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PREFACE 


In  August  1991,  a  major  center  with  emphasis  on  validation  of  nondestructive  inspection  (NDI)  techniques 
for  aging  aircraft  was  established  at  Sandia  National  Laboratories  (SNL)  by  the  Federal  Aviation 
Administration  (FAA).  This  center  is  called  the  Aging  Aircraft  NDI  Development  and  Demonstration 
Center  (AANC).  The  FAA  Interagency  Agreement,  which  established  this  center,  provided  the  following 
summary  tasking  statement:  "The  task  assignments  will  call  for  Sandia  to  support  technology  transfer, 
technology  assessment,  technology  validation,  data  correlation,  and  automation  adaptation  as  on-going 
processes."  Key  to  accomplishing  this  tasking  is  the  FAA/ AANC  Validation  Center,  which  will  reside  in  a 
hangar  leased  from  the  City  of  Albuquerque  at  the  Albuquerque  International  Airport. 

As  one  of  its  first  projects  AANC  established  a  working  group  consisting  of  personnel  from  Sandia, 

Science  Applications  International  Corporation  (SAIC),  and  AEA  Technology.  The  working  group  was 
charged  with  designing  and  implementing  an  experiment  to  quantify  the  reliability  associated  with  airline 
eddy  current  inspections  of  lap  splice  joints. 

The  result  of  AANC's  efforts  is  this  three  volume  document.  Reliability  Assessment  at  Airline  Inspection 
Facilities  which  details  an  experimental  concept  for  inspection  reliability  assessment  and  a  specific 
experiment  designed  to  determine  probability  of  detection  (POD)  curves  associated  with  eddy  current 
inspections.  The  experimental  concept  and  eddy  current  experiment  take  into  account  human  factor 
influences,  which  have  not  been  fully  addressed  in  past  POD  work.  The  result  will  be  a  better 
quantification  of  the  reliability  of  the  inspection  techniques  currently  employed  in  the  field.  This  will  lead 
to  better  inputs  for  damage  tolerance  analysis  and  improved  confidence  in  the  specification  of  inspection 
intervals. 

Because  the  FAA/AANC  NDI  Validation  Center  has  been  tasked  to  pursue  other  related  NDI  reliability 
experiments,  the  protocol  for  this  experiment  was  developed  first  as  a  generic  protocol  then  as  a  specific 
eddy  current  lap  splice  inspection  protocol.  The  generic  protocol  is  presented  in  Vol  I;  A  Generic  Protocol 
for  Inspection  Reliability  Experiments,  and  the  specific  eddy  current  experiment  protocol  is  presented  in 
Vol  II:  Protocol  for  an  Eddy  Current  Inspection  Reliability  Experiment.  Because  of  the  extent  and  duration 
of  the  experiment,  the  actual  results  of  the  experiment  are  presented  separately  in  a  third  volume,  Vol  HI: 
Results  of  an  Eddy  Current  Inspection  Reliability  Experiment. 

The  experiment  proposed  in  this  Volume  will  not  only  result  in  realistic  POD  curves  for  the  eddy  current 
inspection  of  lap  splices,  but  will  also  address  a  fundamental  element  in  the  long  term  goals  of  the 
FAA/AANC  NDI  Validation  Center.  Inspection  validation  includes  the  independent  and  quantitative 
evaluation  of  the  effectiveness  and  reliability  of  an  inspection  methodology.  The  realism  of  the  evaluation 
depends  upon  the  extent  to  which  the  influences  of  inspection  environments  and  other  human  factor 
conditions  arc  incorporated.  The  full-scale  test  bed  in  a  hangar  environment  that  is  being  established  with 
the  FAA/AANC  NDI  Validation  Center  will  be  a  major  step  in  allowing  realistic  evaluations  to  take  place. 
A  missing  ingredient,  however,  is  the  variation  induced  into  the  measurement  process  because  of  the 
differences  in  environmental  and  management  conditions  that  exist  between  facilities.  The  facility  variation 
quantified  from  the  lap  splice  experiment  will  thus  provide  data  that  can  be  used  in  future  validation 
efforts. 
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NOMENCLATURE 


a,  flaw  size 

AANC 

AD 

ANOVA 

a,  p 

baseline 

dress  rehearsal 

ET 

FAA 

false  alarm 

inspector 
lap  splice  joint 

MANOVA 

monitor 


Physical  dimension  of  a  flaw  used  in  POD  analysis  models, 
usually  a  crack  length. 

Aging  Aircraft  NDI  Development  and  Demonstration  Center  run 
by  Sandia  National  Laboratories  for  the  Federal  Aviation 
Administration 

Airworthiness  Directive 

Analysis  of  Variance,  a  statistical  procedure  for  comparing  the 
variability  between  groups  to  the  variability  within  groups. 

Intercept  and  slope  coefficients  used  in  logistic  regression 
equation.  Other  forms  of  the  regresion  equation  may  be 
parameterized  by  p  and  a,  the  mean  and  standard  deviation. 

Set  of  measurements  performed  under  laboratory  conditions  on  a 
given  set  of  flaw  specimens. 

Inspection  tasks  performed  according  to  established  protocols  for 
the  purpose  of  testing,  in  a  realistic  environment,  all  the  functions 
required  in  fielding  a  reliability  assessment  experiment. 

Eddy  current  testing 

Federal  Aviation  Administration 

An  NDI  response  of  having  detected  a  flaw  but  at  an  inspection 
location  where  no  flaw  exists. 

Person  who  applies  an  NDI  technique,  interprets  the  results,  and 
makes  a  judgment  as  to  the  presence  or  absence  of  a  flaw. 

Area  where  different  sections  of  the  outer  fuselage  skin  are  joined. 
The  joint  is  formed  by  one  piece  overlapping  the  other  and  has 
fasteners  installed  through  the  two  layers  of  skin. 

Multivariate  Analysis  of  Variance,  a  statistical  procedure  that 
generalizes  the  analysis  of  variance  to  consider  two  or  more 
dependent  variables  simultaneously. 

Person  tasked  with  observing  and  documenting  the  results  of 
inspections  performed  during  an  NDI  reliability  assessment 
experiment. 


NDI 

Nondestructive  inspection.  Visual  inspection  is  customarily 
excluded  from  being  considered  NDI. 

NTSB 

National  Transportation  Safety  Board 

POD,  POD(a) 

Probability  of  detection.  As  a  function  of  flaw  size,  it  is  the 
fraction  of  flaws  of  nominal  size,  a,  that  is  expected  to  be  detected. 

POFA 

Probability  of  false  alarm 

protocols 

Set  of  written  procedures  for  conducting  all  activities  related  to  the 
implementation  of  a  reliability  assessment  program. 

ROC 

Receiver  Operating  Characteristic,  a  curve  incorporating  detection 
probabilities  with  probabilities  of  false  alarms. 

SAIC 

Science  Applications  International  Corporation 

SID 

Supplemental  Inspection  Documents, 

SNL 

Sandia  National  Laboratories 
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EXECUTIVE  SUMMARY 


This  document  presents  the  protocol  for  an  experiment  to  assess  the  reliability  of  high-frequency  eddy 
current  inspections  of  lap  splice  joints  in  airline  maintenance  and  inspection  facilities.  The  protocol  follows 
the  guidelines  set  forth  in  "Reliability  Assessment  at  Airline  Inspection  Facilities,  Vol.  I.  A  Generic 
Protocol  for  Inspection  Reliability  Experiments,"  DOT/FAA/CT-92/12,1. 

Following  the  Aloha  accident  in  April  1988,  the  National  Transportation  Safety  Board  stated:  "There  are 
human  factor  issues  associated  with  visual  and  nondestructive  inspection  which  can  degrade  inspector 
performance  to  the  extent  that  theoretically  detectable  damage  is  overlooked."  The  goal  of  the  proposed 
experiment  is  to  provide  a  quantitative  assessment  of  inspectors'  performance  in  airline  facility  use  of 
high-frequency  eddy  current  inspection  procedures.  The  "human  factor  issues"  are  thereby  encompassed  in 
the  experiment. 

Secondary  objectives  are  established  in  an  attempt  to  understand  the  lectors  that  influence  the  reliability 
and  to  provide  self-assessment  with  respect  to  the  experimental  plan.  Specific  factors  that  are  to  be  studied 
within  the  experimental  plan  are  (1)  off-angle  cracks,  (2)  unpainted  versus  painted  surfaces,  (3)  variation  of 
reference  standards,  (4)  accessibility  of  task,  (5)  time  into  inspection  (related  to  "boredom"),  (6)  work 
shifts,  and  (7)  differences  in  specimen  definition.  During  the  course  of  the  experiment,  data  will  also  be 
collected  on  conditions  that  exist  at  the  facilities,  such  as  lighting  levels  and  other  environmental  factors. 
Data  specific  to  the  inspectors,  such  as  level  of  training,  experience  level,  and  recentness  of  experience  on 
the  specific  task,  will  also  be  collected. 

Details  are  provided  on  the  two  types  of  test  samples  that  provide  the  test  vehicle.  The  experiment  includes 
36  specimens  (20  by  20  inches)  that  are  configured  as  a  lap  splice  with  specified  flaw  counts  and  flaw 
lengths.  This  represents  720  inspection  sites  (161  flawed)  and  an  estimated  time  of  2  to  2.5  hours  for  each 
inspection.  The  experiment  also  includes  two  Foster-Miller  panels  (10  by  6  feet)  that  will  contain  all  the 
structural  components  found  on  the  aircraft  fuselage.  These  panels  contain  216  inspection  sites  and  should 
take  from  I  to  1.5  hours  to  inspect.  Flaw  orientations  and  densities  will  not  be  controlled  on  these  panels, 
but  a  goal  of  15  to  25  flaws  per  panel  in  the  0.04  to  0.20  inch  has  been  established. 

A  baseline  laboratory  assessment  will  be  conducted  so  that  the  total  effect  of  moving  from  laboratory  to 
field  environments  can  be  evaluated. 
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1.  Introduction 

This  protocol  details  the  procudures,  hardware,  and  techniques  used  to  field  an  experiment  to  quantify  the 
reliability  of  high-frequency  eddy  current  inspections  of  aircraft  lap  splice  joints,  as  the  inspections  are 
performed  in  airline  maintenance  and  inspection  facilities.  (A  schematic  of  a  lap  splice  joint  in  the  outer 
skin  of  the  aircraft  fuselage  is  shown  in  Figure  2-1.)  The  presentation  parallels  the  format  established  in 
"Reliability  Assessment  at  Airline  Inspection  Facilities,  Vol.  I:  A  Generic  Protocol  for  Inspection 
Reliability  Experiments"  [1]. 

1.1  Scope 

The  planned  experiment  is  specific  to  inspection  procedures  used  on  Boeing  737  lap  splice  joints. 

Inspection  procedures  are  given  in  Boeing  Documents  737  D6-37239,  part  6,  subject  53-3-03  and  737 
D6-37239,  part  6,  subject  53-3-05.  Three  inspection  methods  are  covered,  sliding  probe,  oversize 
template,  and  rotating  surface  probe.  Both  the  sliding  probe  and  the  rotating  surface  probe  procedures 
require  that  any  flaw  indication  be  checked  with  the  oversize  template  method.  The  equipment  needed  for 
each  of  these  methods  varies.  The  sliding  probe  method  requires  an  instrument  capable  of  an  impedance 
plane  display.  The  oversize  template  method  can  be »  sed  with  meter  signal  display  instruments,  and  the 
rotating  surface  probe  method  requires  an  oscilloscope  display. 

Inspections  of  riveted  joints  using  high-frequency  eddy  current  technology  are  not  unique  to  Boeing. 
Douglas  and  Airbus  maintenance  manuals  require  the  same  type  of  inspection  using  similar  equipment. 

The  results  of  the  proposed  experiment  will  be  reflective  of  the  technology  as  employed  in  field  conditions 
and  thus  will  be  applicable  beyond  the  single  aircraft  type  chosen  to  provide  definition  for  the  test  vehicle. 

1.2  Background 

Inspection  of  aging  aircraft  is  a  matter  of  national  concern.  Programs  have  been  established  by 
manufacturers,  airline  operators,  and  the  Federal  Aviation  Administration  (FAA)  to  address  these  concerns. 
Specifically,  the  FAA  pu'T^acd  requirements  in  1981  directing  the  development  of  Supplemental 
Inspection  Documents  (SIDs).  The  intent  of  the  SIDs  is  to  extend  routine  maintenance  programs  in  order 
to  detect  fatigue  damage  in  older  aircraft.  High-frequency  eddy  current  inspections  are  an  integral  part  of 
both  routine  maintenance  checks  and  SID  directed  checks  for  surface  fatigue  cracks. 

In  April  1988,  an  Aloha  Airlines  Boeing  737-200  experienced  an  explosive  decompression  and  structural 
failure  while  cn  route  to  Honolulu,  HI.  Subsequent  investigation  indicated  that  a  debond  had  occurred  in  a 
lap  joint,  followed  by  extensive  fatigue  cracking  in  adjacent  rivet  holes.  Eddy  current  inspection  of  the 
upper  row  of  rivets  in  the  lap  joint  was  required  by  an  FAA  Airworthiness  Directive.  Following  the 
accident,  the  National  Transportation  Safety  Board  (NTSB)  stated:  "There  are  human  factor  issues 
associated  with  visual  and  nondestructive  inspection  which  can  degrade  inspector  performance  to  the  extent 
that  theoretically  detectable  damage  is  overlooked"  [2]. 

Evaluations  of  the  reliability  of  lap  splice  eddy  current  inspection  procedures  have  been  carried  out  [3]. 

The  issue  raised  by  the  NTSB  has  not  been  addressed,  however,  as  the  existing  evaluations  have  not 
addressed  the  impact  of  field  conditions.  The  objective  of  the  proposed  experiment  is  to  evaluate  the 
reliability  of  eddy  current  inspection  procedures  as  they  are  done  routinely  at  airline  maintenance  and 
inspection  facilities,  thereby  implicitly  incorporating  human  factors  issues  into  the  evaluation. 
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1.3  Reliability  Assessment  Elements 


Figure  1-1  is  a  flowchart  showing  various  elements  that  are  to  be  included  in  the  lap  splice  experiment. 
These  elements  have  been  grouped  into  four  major  phases.  They  are: 

Experiment  plan 

Field  experiment  implementation 

Data  Analysis 

Reporting  (Plan  of  Action) 

Though  each  of  these  phases  will  be  discussed  individually  in  sections  2  through  5,  the  Plan  of  Action 
deserves  some  comment  here.  Early  in  the  planning  phase  a  decision  was  made  maximize  the  productive 
dissemination  of  the  results  of  this  experiment.  This,  of  course,  would  require  the  identification  of 
correctable  problems,  but  does  not  stop  there.  Intcest  groups  must  be  identified  and  the  character  and 
quantity  of  information  distributed  must  be  tailored  to  these  groups  requirements.  This  function  will 
require  the  concerted  efforts  of  both  the  FAA  Technical  Center  staff  and  AANC  working  group. 
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Key  Deliverables 


Data  Analysis 
(POD) 
(ROC) 


Report  * 


Figure  1-1.  Reliability  experiment  outline. 
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2.  Experimental  Plan 

The  experimental  plan  provides  a  mechanism  to  assess  the  reliability  of  high-frequency  eddy  current 
inspections  of  lap  splice  joints  as  those  inspections  are  performed  in  aircraft  maintenance  and  inspection 
facilities.  Facility'  differences  in  procedures,  equipment,  training,  and  environment  are  thus  part  of  the 
system  being  studied.  The  experimental  plan  does  not  control  these  factors  but  calls  for  gathering  relevant 
data.  (For  a  discussion  of  controlled  versus  uncontrolled  factors  see  Volume  I:  A  Generic  Protocol  for 
Inspection  Reliability  Experiments.)  The  data  gathered  will  be  analyzed  as  potential  explanatory  factors  for 
observed  variations  in  inspection  results. 

2.1  Experimental  Design 

Various  factors  that  can  affect  the  inspection  results  are  identified  and  discussed  in  Section  2.1.1. 
Experimental  objectives  that  guide  the  treatment  of  the  various  factors  in  the  experimental  design  arc  given 
in  Section  2. 1 .2.  The  experimental  design  is  determined  by  the  characteristics  of  the  test  samples  covered 
in  Section  2. 1 .3  and  by  the  procedures  used  in  defining  the  inspection  tasks  covered  in  Section  2. 1 .4.  A 
discussion  of  the  character  and  the  number  of  facilities  to  be  visited  is  given  in  Section  2.1.5. 

2. 1. 1  Variables  Affecting  Results 

There  are  many  factors  related  to  specific  facility  characteristics  that  could  impact  reliability  numbers. 

The  i  unt  of  light,  the  level  of  noise,  the  level  of  comfort,  and  the  procedures  and  management  structures 
of  a  facility  could  all  affect  the  inspection  task. 

Inspector  related  factors  include:  the  amount  of  training  in  eddy  current  testing,  the  reccntncss  of  the 
training,  previous  experience  with  Jap  splice  inspection,  and  the  reccntncss  of  that  experience.  Individual 
inspector  traits,  such  as  susceptibility  to  boredom  and  interactions  with  shift  work,  could  also  influence 
results. 

The  NDI  system  under  study  (high-frequency  eddy  currents)  is  general  enough  that  a  wide  range  of 
equipment  with  a  wide  range  of  capabilities  is  employed  in  the  field  Table  2-1  lists  typical  eddv  current 
equipment  and  shows  the  expected  variation  in  a  number  of  characteristics.  The  list  is  only  of  equipment 
employing  scopes.  Eddy  current  equipment  employing  meters  is  also  used  in  inspections. 

In  addition  to  the  above  factors,  which  arc  specific  to  the  inspection  sy  stem,  flawr  characteristics  can  also 
impact  reliability.  The  angle  at  which  a  crack  emanates  from  a  rivet  hole  could  influence  its  detectability. 
The  condition  of  the  lap  splice  rivets  and  the  characteristics  of  the  material  around  the  rivets  could 
influence  eddy  current  readings  and  thereby  impact  the  detection  reliability. 

There  arc  multiple  lap  splice  joints  on  an  aircraft.  The  probability’  of  detecting  flaws  may  depend  on  the 
particular  lap  splice  being  inspected,  because  of  accessibility  and  other  constraints  put  on  the  inspector. 
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Table  2-1.  Characteristics  of  selected  eddy  current  instrumentation 


2.1.2  Experimental  Objectives 

The  major  objective  is  to  assess  the  reliability  of  field  inspections  of  lap  splices  using  high-frequency  eddy 
current  NDI  procedures.  The  emphasis  is  on  inspections  as  they  occur  in  the  airline  maintenance  and 
inspection  facilities. 

Secondary  objectives  are  established  to  help  understand  the  factors  influencing  reliability  and  also  to  assess 
the  experimental  technique.  The  factors  influencing  the  inspection  results  can  be  categorized  into:  (1)  those 
that  are  specific  to  eddy  current  lap  splice  inspections,  and  (2)  those  that  are  more  general  and  would  be 
present  in  any  NDI  reliability  assessment  field  experiment.  Understanding  the  first  category  helps  in 
gaining  a  better  understanding  of  the  lap  splice  inspections  and  is  necessary  if  improvements  are  to  be  made 
to  the  inspection  process.  Knowledge  in  the  second  category  transfers  to  other  NDI  technologies,  thereby 
reducing  the  need  for  a  full  replication  of  an  extensive,  costly  field  experiment,  such  as  the  one  given  here. 

The  following  objectives  have  been  established  for  the  eddy  current  lap  splice  inspection  experiment. 

1.  Assess  Effects  of  Off-angle  Cracks.  Flaw  characteristics  besides  length  are  expected  to  influence 
detectability.  One  such  characteristic  is  the  angle  at  which  the  flaw  occurs  with  respect  to  the  usual  or 
expected  direction  of  crack  growth.  These  off-angle  (from  horizontal)  cracks  have  been  observed  and  it  is 
important  to  determine  the  effect  of  this  factor  under  field  conditions.  [4] 

2.  Assess  the  Effect  of  Inspecting  Painted  Versus  Unpainted  Surfaces.  Procedures  do  not  require 
that  paint  be  removed  when  there  are  two  or  fewer  layers  and  rivet  locations  can  be  observed.  Thus, 
painted  and  unpainted  surfaces  represent  variations  that  are  likely  to  exist  in  field  inspections. 

3.  Characterize  the  Reference  Standards  Used  Within  a  Facility.  Before  any  inspection  takes  place 
an  inspector  should  calibrate  the  equipment  against  a  reference  standard.  Since  the  reference  standard 
theoretically  sets  a  detection  threshold  based  on  the  length  of  flaw  contained  within  it,  variation  in  reference 
flaws  can  represent  a  major  source  of  inspection  variation.  Characterization  of  the  reference  standards 
provides  a  quantification  of  this  source  of  inspection  variation. 

4.  Effects  of  Accessibility.  There  are  multiple  lap  splices  around  a  fuselage.  Each  one  can 
represent  different  inspection  conditions  according  to  how  accessible  the  inspection  sites  are  and  according 
to  the  physical  demands  placed  on  the  inspector.  It  is  thus  desirable  to  reflect  a  range  of  accessibility  in  the 
experiment  to  gage  the  potential  impact  of  this  factor. 

5.  Assess  Inspection  Time  Effects.  Inspectors  arc  often  asked  to  perform  inspections  over  an 
extended  period  of  time.  It  is  thought  that  "boredom"  could  come  into  play  and  that  the  reliability  of  the 
inspections  performed  initially  could  differ  markedly  from  those  performed  well  into  the  task. 

6.  Gather  Facility  Specific  and  Inspector  Specific  Data  as  Potential  Explanatory  Factors.  The 
facility  specific  and  inspector  specific  factors  will  not  be  included  in  the  set  of  experimentally  controlled 
variables.  However,  the  information  on  these  variables  will  be  gathered  and  analyzed,  as  they  arc  believed 
to  affect  the  reliability  of  inspections.  Light  levels  will  be  measured  at  the  time  of  inspections.  Although 
not  prespecified,  if  information  is  available  about  the  experience  level  and  the  rcccntncss  of  experience  for 
the  pool  of  potential  inspectors,  it  will  be  used  to  obtain  a  breadth  of  these  factors  in  the  overall  experiment. 
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(Objectives  5  and  6  are  not  only  for  the  specific  task  of  lap  splice  inspection,  but  also  will  provide 
information  that  should  be  transferable  to  other  NDI  assessments.  The  following  three  objectives  are 
established  to  obtain  additional  information  that  transfers  to  other  NDI  reliability  assessments.) 

7.  Provide  Baseline  (Laboratory  Environment)  Inspection  Reliability  Assessments.  A  laboratory 
baseline  is  essential  for  providing  a  fiducial  point  for  determining  the  total  effect  of  human  factors  and  field 
conditions  on  the  reliability  assessments. 

8.  Assess  Effects  Connected  with  Shift  Work.  It  is  likely  that  many  of  the  facilities  visited  will  have 
inspection  activities  spread  across  several  shifts.  If  shift  effects  exist,  they  are  likely  to  apply  to  other 
inspection  tasks. 

9.  Assess  the  Effect  of  Specimen  Definition.  Specimens  used  in  field  experiments  can  be  designed 
and  fabricated  based  on  various  rationales.  Controls  on  flaw  sizes,  distributions,  and  orientations  can  be 
imposed  for  statistical  design  reasons,  or  flawed  specimens  can  be  designed  to  simulate  "naturally" 
occurring  flaws.  Any  differences  in  the  estimates  provided  are  artifacts  of  the  experimental  techniques. 
Approaches  for  future  reliability  studies  would  be  influenced  by  the  knowledge  of  any  differences. 

2. 1.3  Experimental  Design  of  Test  Samples 

Two  types  of  test  samples  will  be  used  in  the  experiment.  One  type  will  be  20  inches,  by  20  inches, 
specimens  fabricated  to  simulate  fuselage  lap  splice  joints  on  the  Boeing  737.  The  other  type  of  test  sample 
will  be  10  ft  by  6  ft  panels,  produced  with  all  of  the  structural  components  found  in  an  aircraft  fuselage. 
These  panels  will  include  two  longitudinal  lap  splice  joints  and  will  be  curved  to  match  the  nominal  75 
inches,  radius  found  on  the  Boeing  737. 

In  all  of  the  test  samples,  the  lap  splice  being  simulated  will  have  three  rows  of  rivets.  Only  the  top  row  of 
rivets  will  contain  flaws  and  be  inspected  in  compliance  with  FAA  AD  88-22-1 1 . 

Two  types  of  test  samples  are  included  to  address  Objective  9.  The  smaller  specimens  w  ill  have  flaw  s  that 
arc  controlled  in  placement  and  in  distribution.  The  larger  panels  will  be  more  "natural"  in  the  flaw 
characteristics.  Details  of  the  fabrication  are  given  in  Sections  2.2. 1  and  2.2.2  and  in  Appendix  B. 

Thirty  six  of  the  small  specimens  will  be  used  in  this  experiment,  providing  720  rivet  inspection  sites.  Of 
these,  559  will  be  unflawcd  sites  and  161  will  be  flawed  sites.  The  specimens  will  be  mounted  end-to-end 
on  a  frame  such  that  they  simulate  two  30-foot  lengths  -  there  will  be  an  upper  and  lower  row  of  aircraft 
fuselage  lap  splice  joints.  Both  the  frame  and  the  specimen  mounting  arc  described  in  detail  in  Test 
Framc/Support  Structure  Section  2.2.3  and  Appendix  B. 

Each  row  represents  a  different  level  of  accessibility  and  mode  of  operation  with  respect  to  the  interactions 
between  the  inspector,  the  inspection  site,  and  the  inspection  equipment.  The  top  row  can  be  inspected  in  a 
standing  position.  The  bottom  row  will  require  the  inspector  to  crouch.  Thus,  Objective  4  is  addressed. 

Each  of  the  small  specimens  will  contain  fatigue  cracks  at  one  of  three  density  levels.  The  crack  density 
levels  for  each  panel  are  "none",  "low"  (1  to  3  holes  with  cracks),  and  "high"  (7  to  9  holes  with  cracks). 
Crack  lengths  will,  for  the  most  part,  be  in  the  range  of  0.04  inches,  to  0.20  inches.  Some  cracks  as  small 
as  0.02  inches,  will  also  be  included.  One  half  of  the  cracks  will  emanate  from  the  left  side  of  the  holes 
while  half  will  emanate  from  the  right  side.  The  placement  will  be  randomly  determined.  Thirty  two  of  the 
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rivet  locations  will  contain  two  cracks  of  roughly  the  same  magnitude  that  will  emanate  from  both  sides  of 
the  rivet  hole. 


Including  rivet  sites  with  cracks  from  both  sides  of  the  hole  will  enable  a  test  of  the  hypothesis  that  the 
probability  of  detection  associated  with  the  rivet  can  be  derived  by  treating  each  of  the  cracks 
independently.  Questions  such  as  this  are  important  in  determining  the  proper  analysis  for  multiple  site 
damage  scenarios  when  using  POD  curves.  One  of  the  specimens  containing  nine  cracked  rivet  sites  will 
also  contain  a  1-in.  crack  that  extends  from  hole  to  hole.  We  expect  that  all  inspectors  will  find  this  crack. 

In  addition  to  crack  density,  each  small  specimen  will  also  possess  one  of  three  levels  of  "off-angleness"  for 
the  direction  of  crack  growth.  The  three  levels  are:  (1)  0°  off  horizontal  (i.e.  along  hole  centerline),  (2)  1 1° 
off  horizontal,  and  (3)  22°  off  horizontal.  Having  three  levels  provides  data  for  characterizing 
"non-linearity"  in  the  effect  of  crack  growth  direction  on  detection  reliability.  (See  Section  2.4  Data 
Analysis).  The  maximum  of  22°  for  the  off-angle  cracks  is  based  on  reported  occurrences  at  this  level.  [4] 

Table  2-2  gives  the  intended  distribution  of  cracks,  including  the  crack  lengths  and  direction  of  growth. 
Tabie  2-3  shows  the  specimen  distribution  with  respect  to  the  number  of  cracks  per  specimen  and  the 
direction  of  crack  growth.  The  total  number  of  cracked  rivets  are  also  listed  in  these  tables. 


Table  2-2.  Desired  distribution  of  cracks  by  length  of  crack  and  crack  direction 


Crack  Length  (in.) 

0° 

Crack  direction 

11° 

22° 

Total 

.02  to  .04 

7 

2 

2 

11 

.04  to  .  12 

66 

20 

20 

106 

.12  to  .20 

26 

8 

8 

42 

1  * 

1 

1 

Totals 

100 

30 

30 

160 

*  One  crack  will  extend  from  one  rivet  hole  to  the  next. 


The  larger  panels  allow  cracks  to  grow  "naturally,"  with  no  controls  on  their  orientation  or  relationship  to 
each  other.  However,  getting  meaningful  POD  estimates  will  require  that  cracks  be  produced  in  the  .04 
inches,  to  .20  inches,  range.  The  objective  is  to  produce  two  of  these  large  panels  and  to  have  15  to  25 
flawed  sites  per  panel  in  the  .04  to  .20  inches,  range.  A  total  of  108  rivcts/pancl  will  be  inspected. 

Specimen  Finish  -  The  Boeing  procedures  for  inspecting  lap  splice  joints  call  for  the  stripping  of  paint  if 
there  are  more  than  two  layers  or  if  rivet  heads  cannot  be  visually  located.  However,  a  particular  facility 
may  strip  the  paint  as  a  matter  of  procedure.  The  small  specimens  will  be  painted  or  unpainted  according 
to  the  facility's  normal  mode  of  operation.  One  of  the  large  panels  will  be  painted  and  one  unpainted 
throughout  the  experiment. 
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Table  2-3.  Panel  distribution  by  desired  number  of  cracked  sites  and  crack  direction.  Also  shown  is  the 

total  number  of  cracked  rivets  ( ). 


Crack  direction 


Number  of  Cracked  0  0 

sites  /Panel 


low  density  1 

3  (3) 

2 

4  (8) 

3 

3  (9) 

high  density  7* 

3  (21) 

8* 

4  (32) 

9* 

3  (28) 

Totals 
Uncracked  panels 

20(101) 

11° 

22° 

Totals 

1  0) 

1  0) 

5  (5) 

1  (2) 

1  (2) 

6  (12) 

1  (3) 

I  (3) 

5  (15) 

1  (7) 

1  (7) 

5  (35) 

1  (8) 

1  (8) 

6  (48) 

1  (9) 

1  (9) 

5  (46) 

6(30) 

6(30) 

32(161) 
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*  Some  rivets  in  each  panel  will  contain  cracks  from  both  sides  of  the  hole  (see  Figure  2-1).  One  of  the  9-crack 
panels  will  have  a  1  inches,  crack  extending  to  an  adjacent  rivet.  Thus,  one  additional  rivet  is  included  in  the  9- 
crack  panels. 


A  practical  consideration  related  to  the  specimen  finish  is  the  amount  of  surface  damage  that  results  when 
the  probe  contacts  the  surface  being  inspected.  The  unpainted  surfaces  will  become  scratched  and  paint  will 
be  removed  from  the  painted  surfaces,  which  may  be  a  problem  in  areas  where  the  inspectors  spend  more 
time  because  of  flaw  indications.  This  is  unacceptable  since  the  circular  mars  on  the  surface  will  provide 
unrealistic  visual  clues  to  subsequent  inspectors. 

Two  methods  to  eliminate  these  visual  clues  are  considered.  One  is  to  cover  the  row  of  rivets  with  a  thin, 
transparent  tape  that  would  act  as  a  surface  protector.  The  tape  would  also  allow  the  inspector  to  mark  the 
test  specimens  and  would  provide  a  semi-permanent  record  of  each  inspection  to  help  eliminate  data 
recording  errors.  A  second  alternative  is  to  repaint  or  rebuff  the  panels  whenever  the  surface  damage  is 
excessive. 

A  small-scale  experiment  consisting  of  rotating  different  types  of  probe  tips  on  differently  protected 
surfaces  was  performed  on  an  aluminum  aircraft  wing  section.  The  variables  investigated  were:  (1)  type  of 
probe  tip  -  plastic,  aluminum,  steel,  and  "sharp"  (a  .025  inches,  mechanical  pencil  tip);  (2)  surface  -  bare 
aluminum,  painted  aluminum,  bare  aluminum  buffed  after  test,  painted  aluminum  protected  with  three 
types  of  Scotch  transparent  tape,  (3)  probe  tip  revolutions  -  20  or  200  revolutions  corresponding  to 
approximately  5  and  50  inspections. 

The  results  of  the  experiment  show  that  the  bare  aluminum  is  easily  marked  with  everything  but  the  plastic 
probe  material.  It  is  possible  to  buff  out  the  20  revolution  marks  using  a  commercial  grade  "finishing  pad." 
Indications  are  that  this  would  be  necessary  after  every  one  or  two  inspections  to  eliminate  the  added  visual 
clues.  Over  the  life  of  the  experiment,  it  is  estimated  that  rebuffing  a  bare  aluminum  surface  would  remove 
from  .005  inches,  to  .010  inches,  of  the  surface.  This  could  have  a  major  affect  on  the  repeatability  of  the 
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test  samples  from  the  beginning  to  the  end  of  the  experiment.  Also,  if  painting  were  involved,  the  paint 
drying  time  would  be  added  to  the  time  taken  for  refurbishing.  The  loss  of  repeatability  and  the  added  man 
power,  time,  and  costs  are  enough  to  warrant  using  a  protective  tape. 

There  are  trade-offs  in  using  or  not  using  the  protective  tape.  The  issue  has  been  addressed  with  expert 
input  as  to  the  effect  on  the  inspection  task.  Weighing  all  factors  we  have  decided  to  employ  the  use  of 
tape. 

The  use  of  protective  tape  presents  the  same  surface  condition  to  all  inspections  regardless  of  original 
surface  finish.  The  experimental  protocols  will  call  for  a  monitor  to  cover  the  reference  standard  surfaces 
with  the  same  protective  tape  that  is  used  on  the  test  samples.  The  inspector  will  then  follow  usual 
calibration  procedures;  therefore,  variations  resulting  from  the  treatment  of  the  reference  standard  with 
regards  to  painted  and  unpainted  surfaces  will  still  be  present.  The  inspector  will  also  have  the  same  visual 
presentation  to  which  he  is  accustomed;  therefore.  Objective  2  (page  6)  is  addressed  to  the  extent  indicated. 

2.1.4  Inspection  Procedures 

At  each  facility,  there  will  be  four  basic  layouts  or  arrangements  of  the  small  specimens.  The  basic  layout 
has  the  inspector  going  from  an  initial  inspection  period  with  no  cracks  (about  8  minutes),  to  a  period  of 
inspecting  at  a  relatively  low  flaw  density  (10%),  followed  by  a  period  of  high  flaw  density  (40%).  This 
sequence  is  repeated  with  the  inspection  of  the  other  lap  splice  row. 

The  initial  period  with  no  cracks  is  maintained  in  all  the  layouts.  The  intention  is  to  counter  the  natural 
tendency  of  the  inspector  to  believe  that  the  experimental  apparatus  contains  many  flaws.  The  three 
additional  layouts  are  derived  by  reversing  the  order  of  inspection  on  the  basic  layout  and  by  interchanging 
the  rows  of  panels  (sec  Table  2-4).  With  the  four  layouts,  each  crack  will  appear  in  both  top  and  bottom 
lap  splice  rows  and  will  appear  in  each  of  four  time  periods  of  inspection  in  a  balanced  manner.  This  will 
ensure  the  ability  to  cleanly  estimate  accessibility,  time,  and  flaw  density  effects  (Objectives  4  and  5,  page 
6). 

The  inspectors  will  mark  the  rivet  locations  where  a  flaw  is  detected  directly  on  the  tape  covering  the 
specimen.  If  the  sliding  probe  method  is  used  as  an  initial  screen,  all  indicated  flaw  locations  will  be  noted 
in  the  data  base.  For  the  subsequent  oversize  template  method,  the  inspector  will  be  asked  to  give  a 
three-level  confidence  rating  to  all  flaw  indications.  The  different  ratings  will  be  used  as  criteria  changes 
for  the  purpose  of  determining  ROC  curves. 


Table  2-4.  Layout  of  panels.  Lc,Hc  -  low  and  high  density  cracks  on  centerline 
Le,He  -  low  and  high  density  cracks  at  1 1®.  Lt,Ht  -  low  and  high  density  cracks  at  22®. 
PP,UP  -  painted  and  unpainted  large  curved  panels. 

Each  line  of  a  layout  corresponds  to  a  different  row  of  panels. 


Panel  Position 


lay 

out 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

1 

0 

0 

Lc 

Te 

Lc 

lT 

Lc 

Lc 

Le 

Lc 

He 

Ht 

He 

He 

He 

Ht 

He 

He 

PP 

uF 

0 

0 

Lc 

Lc 

Lt 

Lc 

Le 

Lc 

Lt 

Lc 

He 

He 

He 

He 

Ht 

He 

He 

He 

2 

0 

0 

Lc 

Lc 

Lt 

Lc 

Le 

Lc 

Lt 

Lc 

He 

He 

He 

He 

Ht 

He 

He 

He 

UP 

PP 

0 

0 

Lc 

Le 

Lc 

Lt 

Lc 

Lc 

Le 

Lc 

He 

Ht 

He 

He 

He 

Ht 

He 

He 

3 

0 

0 

He 

He 

Ht 

He 

He 

He 

Ht 

He 

Lc 

Le 

Lc 

Lc 

Lt 

Lc 

Le 

Lc 

PP 
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0 

0 

He 

He 

He 

Ht 

He 

He 

He 

He 

Lc 

Lt 

Lc 

Le 

Lc 

Lt 

Lc 

Lc 

4 

0 

0 

He 

He 

He 

Ht 

He 

He 

He 

He 

Lc 

Lt 

Lc 

Le 

Lc 

Lt 

Lc 

Lc 

UP 

PP 

0 

0 

He 

He 

Ht 

He 

He 

He 

Ht 

He 

Lc 

Le 

Lc 

Lc 

Lt 

Lc 

Le 

Lc 

Inspection  layouts  by  panel  numbers 

lay¬ 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

out 

1 

1 

2 

3 

~4 

T 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

2 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

3 

2 

1 

18 

17 

16 

15 

14 

13 

12 

11 

10 

9 

8 

7 

6 

5 

4 

3 

20 

19 

36 

35 

34 

33 

32 

31 

30 

29 

28 

27 

26 

25 

24 

23 

22 

21 

4 

20 

19 

36 

35 

34 

33 

32 

31 

30 

29 

28 

27 

26 

25 

24 

23 

22 

21 

2 

1 

18 

17 

16 

15 

14 

13 

12 

11 

10 

9 

8 

7 

6 

5 

4 

3 

2. 1.5  Number  of  Facilities  and  Inspections 

Each  of  the  four  layouts  will  be  inspected  by  a  different  inspector.  The  inspectors  will  be  chosen  to 
represent  every  shift  that  the  facility  operates  to  address  Objective  8.  For  those  facilities  with  three  shifts, 
the  fourth  inspector  will  be  chosen  to  balance  the  representation  on  all  three  shifts  across  facilities.  At  all 
facilities,  the  initial  inspector  will  be  asked  to  do  two  inspections.  The  second  inspection  will  be  at  the  end 
of  the  facility  visit.  The  result  will  be  a  total  of  five  inspections  at  each  facility. 

Table  2-5  shows  the  U.S.-based  airlines  that  fly  Boeing  737s  in  their  fleets  as  of  January  1992.  In  addition 
to  the  airlines,  32  independent  repair  stations  capable  of  performing  heavy  airframe  maintenance  are 
identified  in  Table  2-6. (5] 
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Table  2-5.  United  States  Airlines  Operating  Boeing  737  Aircraft 


Airline 

Fleet  Size 

Alaska  Airlines 

7 

Aloha  Airlines 

19 

America  West  Airlines 

65 

American  Airlines 

5 

Carnival  Airlines 

4 

Continental  Airlines 

94 

Delta  Airlines 

72 

Markair 

12 

Midway  Airlines 

1 

Sierra  Pacific  Airlines 

7 

Southwest  Airlines 

123 

Spirit  of  America  Airlines 

1 

United  Airlines 

195 

USAir 

229 

TOTAL 

834 

Source:  Federal  Express  Aviation  Services,  Inc.  Commercial  Jet  Fleets.  Vol.  1 .  Jan.  1 992 


It  is  not  uncommon  for  the  major  airlines  to  have  more  than  one  maintenance  facility.  Thus,  it  is  estimated 
that  there  are  from  30  to  40  total  facilities  where  Boeing  737  lap  splice  inspections  could  be  performed.  To 
have  a  reasonable  chance  (90%)  of  including  at  least  one  out  of  the  most  extreme  10%  of  the  population  in 
a  random  sample,  16  to  17  facilities  should  be  selected.[l]  Rather  than  relying  on  a  totally  random  sample 
to  reflect  the  expected  facility  variation,  classes  of  facilities  can  be  targeted  to  try  and  cover  the  suspected 
major  variation,  which  is  the  tack  recommended  here. 

There  are  many  factors  that  could  be  used  to  characterize  inspection  facilities  with  respect  to  a  specific 
inspection  task.  Some  examples  of  the  factors  are  number  of  times  the  task  is  performed,  proportion  of 
total  work  represented  by  the  task,  independent  or  airline  owned,  inspector  training  programs,  union  or 
non-union  shop,  management  procedures,  and  hangar  conditions  (size,  lighting,  noise,  etc.).  Some  of  the 
many  factors  can  be  associated  with  facilities  prior  to  inclusion  in  the  sample;  others  will  not  be  known 
prior  to  visitation  and  may  not  ever  be  explicitly  known.  To  adequately  cover  the  range  of  possibilities  for 
the  various  variables,  we  recommend  that  12  facilities  be  visited.  These  12  facilities  will  rep  resent  a 
judgement  sample  based  upon  representing  the  range  of  facility  characteristics  that  can  be  known  in 
advance. 
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Table  2-6.  Independent  Maintenance  and  Repair  Stations 

Repair  Station 

Location 

AAR  Oklahoma 

Oklahoma  City,  Ok. 

Aerotest,  Inc. 

Mojave,  Ca. 

Aircraft  Maintenance  Services,  Inc. 

Miami,  Fla. 

Airtech  Service 

Miami,  Fla. 

Associated  Air  Center 

Dallas,  Tx. 

Boeing  Wichita  Company 

Wichita,  Kan. 

Butler  Aviation 

Newark,  NJ 

Chrysler  Technologies  Airborne  Sys. 

Waco,  Tx. 

Commodore  Aviation  Services 

Miami,  Fla. 

Cross  Continent  Aircraft  Services 

Smyrna,  Tn. 

Dalfort  Aviation  Services 

Dallas,  Tx. 

Dee  Howard  Company 

San  Antonio,  Tx. 

DynAir  Tech  of  Arizona 

Phoenix,  A z. 

DynAir  Tech  of  Florida 

Miami,  Fla. 

E-Systems,  Inc. 

Greenville,  Tx. 

Elsinore  Airframe  Services,  Inc. 

Waco,  Tx. 

Evergreen  Air  Center 

Marana,  A z. 

Florida  West  Airlines 

Miami,  Fla. 

Greenwich  Air  Services 

Miami,  Fla. 

Hughes  Aviation  Services 

Las  Vegas,  Nev. 

Intertec  Aviation 

Goodyear,  A z. 

Lockheed  Aeromod  Center 

Greenville,  SC 

Lockheed  Aeromod  Center 

Tucson,  A z. 

Midway  Aircraft  Engineering,  Inc. 

Miami,  Fla. 

NARCAM  Aircraft,  Inc. 

Hialeah,  Fla. 

Page  Aviation,  Inc. 

Orlando,  Fla. 

Pemco  Aeroplex 

Birmingham,  Alabama 

Pemco  Aeroplex-Clearwater 

Clearwater,  Fla. 

Pemco  Aeroplex-Dothan 

Dothan,  Ala. 

Timco 

Greensboro,  NC 

Tracor  Aviation 

Santa  Barbara,  Ca. 

Tramco 

Everett,  Wa. 

2.1.7  Reference  Standard  Data 

Objective  3  is  to  characterize  the  reference  standards  used  to  set  up  the  eddy  current  equipment  within  the 
facilities.  The  NDI  specialist  who  will  be  traveling  with  the  experiment  will  characterize  all  reference 
standards  used  for  setting  up  equipment  used  in  lap  splice  inspections.  The  characterization  will  be  against 
a  master  reference  that  will  travel  with  the  experiment.  The  data  recorded  will  be  variables  data  and  will 
directly  reflect  the  variation  induced  into  the  inspection  process  by  using  different  reference  standards. 
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2. 1. 7  Laboratory  Baseline 


Objective  7  was  to  provide  a  baseline  for  determining  the  total  effect  of  the  human  factors  and  field 
conditions  on  the  reliability  assessments.  This  will  be  done  by  having  four  inspections  of  the  experimental 
specimens  by  four  different  inspectors  prior  to  the  facility  visits.  The  inspections  will  be  done  in  a 
laboratory  setting  with  no  time  pressures  on  the  inspectors.  One  of  the  inspectors  will  be  asked  to  do  two 
inspections,  with  the  inspections  separated  in  time  by  several  days.  In  terms  of  analysis,  the  laboratory 
baseline  becomes  another  "facility".  The  aggregate  laboratory  data  can  then  be  compared  directly  against 
the  data  from  each  of  the  facilities. 

2.2  Design  and  Manufacture  of  Experimental  Specimens 

Two  types  of  test  samples  will  be  employed  in  this  experiment.  One  type  is  test  panels  that  will  be  able  to 
be  moved  and  presented  according  to  the  layouts  presented  in  Table  2-4.  The  other  type  is  panels  that 
incorporate  fuselage  structure  into  the  sample  before  fatigue  cracks  are  grown.  Each  is  explained  more 
fully  below. 

2.2.1.  Lap  Splice  Joint  Test  Specimens 

The  test  panels  used  in  this  experiment  will  simulate  fuselage  lap  splice  joints  found  on  the  Boeing  737 
aircraft.  They  consist  of  two  plates  fastened  together,  using  three  rows  of  rivets,  with  a  3  inch  overlap.  The 
upper  plate  will  contain  the  cracked  holes  in  the  critical  first  row  of  rivets.  This  panel  specimen  is  shown  in 
Figure  2-1. 

Panel  Geometry.  The  completed  panel  will  measure  20  inches  by  20  inches.  These  dimensions  were 
arrived  at  after  careful  consideration  was  given  to  both  the  aircraft  structure  being  modeled  and  the 
maximum  plate  size  that  could  be  economically  produced  without  sacrificing  the  ability  to  generate  the 
desired  flaw  distribution.  The  20  inch  width  allows  the  panels  to  be  mounted  on  a  typical  aircraft 
circumferential  rib  spacing  of  20  inches.  The  20  inch  width  also  simulates  the  aircraft  fuselage  in  the  local 
area  where  the  inspection  task  takes  place.  When  panels  of  this  size  are  assembled  on  a  test  stand,  they  will 
provide  a  good  representation  of  the  longitudinal  lap  splice  joint.  The  method  used  to  produce  the  flaws  in 
the  upper  plate,  along  with  the  requirement  for  off-angle  cracks,  are  the  limiting  factors  in  the  final  height 
of  the  panel.  These  issues  are  described  in  Appendix  B. 

Specimen  Manu  facture.  The  test  panel  production  is  made  up  of  two  distinct  tasks.  One  task  is  the  crack 
initiation  and  growth  and  the  other  task  is  riveting  the  plates  together  to  produce  the  final  test  specimen. 

The  most  difficult  aspect  of  constructing  these  panels  is  the  production  of  the  upper  cracked  plates  where 
the  crack  distribution  is  very  specific  in  number,  length  and  orientation.  Numerous  companies  (Control 
Tech,  Mctcut  Industries,  Teledyne  Engineering,  Foster-Miller  Corp.,  Martin  Marietta,  Cortest,  SNL,  AEA) 
with  expertise  in  this  area  were  contacted  in  order  to  obtain  information  regarding  the  preferred  method  of 
fabrication  and  also  to  obtain  time  and  cost  estimates.  Each  company  that  felt  capable  of  producing  these 
plates  determined  that  a  tension-tension  loading  was  the  best  way  to  generate  cracks  in  the  upper  plate  of 
the  lap  splice  joint.  More  detail  is  given  in  Appendix  B. 
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Material:  Clad  Aluminum  2024-T3 


2.2.2.  Full-size  Aircraft  Panels 


To  provide  test  specimens  with  more  "natural"  crack  distributions;  that  is,  panels  that  have  not  used  starter 
notches  or  controlled  loading  to  achieve  a  designed  crack  distribution,  the  POD  Experiment  will  also 
include  two  full-size  aircraft  panels.  With  these  panels  it  will  be  possible  to:  (1)  provide  more  realism  both 
in  the  size  of  the  panels  and  the  presence  of  "real"  crack  distributions  and  (2)  test  whether  differences  exist 
in  the  estimates  of  POD  curves  when  using  smaller  specimens  with  more  controlled  crack  distributions 
versus  larger  panels  that  contain  natural  crack  distributions. 

Aircraft  Test  Panel  Overview.  These  full-size  panels  will  be  produced  with  all  of  the  structural 
components  found  on  an  aircraft  fuselage,  will  include  one  longitudinal  lap  splice  joint,  and  will  be  curved 
to  match  the  75  inch  radius  found  on  the  Boeing  737  aircraft.  The  panels  will  be  fabricated  by 
Foster-Miller,  Inc.,  and  will  be  similar  to  the  panels  which  Foster-Miller  has  produced  for  other  FAA 
investigations,  (see  Figure  2-2).  Cracks  will  be  generated  in  the  panels  using  a  custom  designed  load 
machine  that  Foster-Miller  developed  for  this  purpose.  The  structural  test  frame  provides  a  bi-axial  load 
(hoop  stress  and  axial  stress)  that  simulates  the  fuselage  loads  incurred  during  aircraft  pressurization.  The 
loads  arc  applied  in  a  cyclic  manner  and  cracks  arc  allowed  to  initiate  as  they  would  in  high  cycle  aircraft. 

A  complete  description  of  the  Foster-Miller  panels,  the  load  facility,  and  the  initial  test  program  carried  out 
in  this  facility  is  provided  in  Ref.  [6],  A  summary  follows. 

Full-Size  Test  Panel  Design  and  Fabrication.  The  test  panel  is  designed  to  be  more  realistic  than  the  small 
specimens  both  in  size  and  features,  which  include  the  following: 

1  Fuselage  construction  like  the  Boeing  737. 
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2.  Thin-skinned  (0.040  inch  thick)  shell  using  clad  2024-T3  aluminum  alloy. 

3.  Stiffening  in  the  circumferential  direction  by  ribs  and  longitudinally  by  stringers. 

4.  Stringers  and  frames  fabricated  from  7075-T6  aluminum  alloy  sheet. 

5.  Rivet  construction  that  adheres  to  Boeing  aircraft  specifications. 

6.  Skin  panels  joined  by  a  lap  joint  using  three  rows  of  countersunk  rivets. 

2.2.3.  Test  Frame/Support  Structure 

The  support  frame  must  adequately  model  the  geometry  of  the  chosen  inspection  area.  The  frame  should 
also  contain  all  of  the  features  that  are  necessary  to  assure  inspector  "buy-in."  The  specimen  support  frame 
for  the  experiment  is  designed  to  hold  the  small  test  specimens  as  well  as  the  full-scale  aircraft  panels.  It 
was  also  designed  for  ease  of  assembly,  ease  of  shipping,  and  for  repetitive  alignment  of  the  test  specimens. 
These  features  are  especially  important  since  this  experiment  requires  the  frame  to  be  assembled  and 
disassembled  several  times.  When  the  support  frame  is  completely  assembled  with  the  test  specimens  it  will 
model  a  total  of  50  ft  length  of  an  aircraft  fuselage.  Thirty  feet  of  this  "fuselage"  will  consist  of  the  20  inch 
wide  test  specimens  mounted  end-to-end  on  the  frame.  The  remaining  20  ft  will  be  comprised  of  the  two  10 
ft,  full-scale,  Foster-Miller  panels. 

The  support  frame  is  designed  for  use  beyond  this  experiment  and  will  be  used  in  the  FAA/AANC 
Validation  Center  following  the  completion  of  the  experiment. 

Further  details  of  frame  design  are  given  in  Appendix  B. 

2.3  Characterization  of  Lap  Splice  Specimens 

The  purpose  of  specimen  characterization  is  to  confirm  the  size  of  the  cracks  and  subsequently  enter  the 
information  into  a  data  base.  The  characterizations  will  be  done  following  the  manufacture  of  the 
specimens. 

Candidate  techniques  that  may  be  used  include  laser  profiling  and  liquid  penetrants.  Laser  profiling  allows 
the  use  of  automation  and  the  obtaining  of  images  that  can  be  processed  and  displayed.  Laser  profiling  can 
be  done  using  Rodcnstock  Precision  Optics,  and  the  results  will  be  in  the  form  of  scan  disks  from  which  the 
data  can  be  extracted  and  color  images  made.  Liquid  penetrants  are  standard  in  detecting  and  measuring 
surface  cracks.  Liquid  penetrant  testing  is  likely  to  be  done  using  FP-99A  water  washable  fluorescent 
penetrant  and  D-701  developer,  manufactured  by  Met-L-Chek.  These  materials  exhibit  minimum  sulphur 
and  halogen  content  per  OSHA  requirements.  They  also  do  not  require  special  disposal  considerations. 

The  FP-99a  has  a  level  3++  sensitivity  rating  providing  good  apability  for  sizing  the  cracks  contained  in 
the  panels.  The  test  results  would  be  photographs  of  each  plate  while  it  is  under  the  black  light. 

Eddy  current  imaging,  plastic  replication,  and  ultrasound  techniques  are  other  methods  of  measurement  for 
consideration  to  increase  the  confidence  in  the  results. 

Capability  for  recharacterizing  the  samples  will  be  maintained  throughout  the  experiment. 
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2.4  Data  Analysis 


The  data  will  be  analyzed  for  POD  through  the  log  odds  model.  [1]  The  parameters  (a  and  p)  in  that 
model  can  be  expanded  to  reflect  a  dependence  on  various  explanatory  factors.  The  basic  model  equation 
that  reflects  the  controlled  factors  of  the  experiment  on  the  small  test  specimens  is  given  by 

ln[POD(a)/(  1  -PGD(a))]  =  F  +  S  +  A  +  D  +  T  +  V  +  p-ln(a), 

where  a  is  the  crack  length,  F  reflects  a  facility  average,  S  reflects  the  shift  (three  levels),  A  is  the  factor  for 
accessibility  (two  levels),  D  is  the  factor  for  the  density  of  flaws  (two  levels),  T  is  the  factor  for  the  time 
into  the  task  (four  levels),  and  V  reflects  different  flaw  angles  (three  levels).  The  experimental  design 
allows  for  not  only  the  estimates  of  each  of  the  main  effects  given  above,  but  also  allows  for  estimating 
most  of  the  interaction  effects. 

Although  not  reflected  in  the  above  model,  interaction  effects  between  the  given  factors  and  p  will  also  be 
analyzed. 

The  basic  model  equation  for  the  large  panels  is  given  by 

In[POD(a)/(  I  -POD(a))]  =  F  +  S  +  P  +  p-ln(a), 

where  F  and  S  arc  facility  and  shift,  respectively,  and  P  is  the  painted/unpainted  factor.  (Note:  the 
painted/unpainted  factor  was  not  included  in  the  model  for  the  small  specimens  because  of  the  uncertainty 
about  the  balance  between  the  facilities  performing  inspections  on  painted  versus  unpainted  surfaces.) 

A  comparison  of  the  composite  a's  and  p's  from  each  of  the  test  types  will  be  analyzed  for  indications  of 
differences  because  of  the  type  of  specimen  being  inspected. 

The  ROC  curve  that  incorporates  false  calls  into  the  analysis  will  be  determined  from  the  three  confidence 
levels  that  the  inspectors  arc  to  provide  with  each  positive  flaw  response.  Empirical  ROC  curves  can  be 
determined  for  each  inspection  and  condition  within  each  inspection  (such  as  crack  angles).  Summary 
measures  can  then  be  determined  from  these  individual  ROC  curves  (sec  Ref.  [  1  ]).  For  example,  one 
accepted  index  for  summarizing  ROC  curves  is  the  area  under  the  curve,  denoted  "A."  This  index  is  an 
average  probability  of  detection,  where  the  average  is  taken  over  the  criteria  range.  The  criteria  range 
corresponds  to  changing  the  threshold  for  calling  a  signal  a  flaw.  The  index  "A"  will  be  modeled  in  the 
same  manner  as  was  the  log  odds  for  the  POD  curves. 

The  index  "A"  will  also  be  determined  for  each  of  several  groups  of  similar  crack  lengths.  This  enables  an 
assessment  with  respect  to  signal  strength  (crack  length). 

The  models  discussed  above  do  not  contain  variables  related  to  uncontrolled  factors.  Observations  of  many 
of  these  variables  will  take  place  at  the  time  the  inspections  occur  and  their  values  will  become  part  of  the 
data  base.  Exploratory  Analysis  of  Variance  (ANOVA)  and  regression  models  will  be  pursued  with 
respect  to  these  factors  in  both  the  POD  and  the  ROC  curves.  The  extension  of  ANOVA  into  Multivariate 
Analysis  of  Variance  (MANOVA)  occurs  when  more  than  one  response  variable  is  analyzed  at  a  time;  for 
example,  simultaneously  analyzing  a  and  p  in  the  log  odds  model  Similarly,  if  two  or  more  responses  are 
modeled  with  the  same  set  of  explanatory  factors  in  a  regression  setting,  the  techniques  of  multivariate 
regression  analysis  can  be  used.  These  techniques  will  be  used  for  analyzing  the  data 


2.5  Protocols 


The  protocols  are  designed  to  run  the  whole  experiment  consistently  covering  interactions  with  the 
facilities,  actions  of  the  participants  and  data  analysis.  A  flowchart  of  activities  taking  place  at  each 
facility  is  given  in  Figure  2-3. 


Figure  2-3.  Experiment  Procedure. 

The  personnel  responsible  for  the  on-site  implementation  of  the  experiment  are  termed  monitors.  Their 
protocols  define  what  actions  the  monitors  will  take  during  the  experiment,  including  briefings  to  the 
facility  management  and  inspectors. 

Similarly,  a  basic  set  of  protocols  for  the  inspector  is  required  to  define  the  procedures  he  will  use  to  carry 
out  the  experiment  and  the  modifications  necessary  to  run  the  experiment. 
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These  protocols  will  be  drafted  prior  to  a  dress  rehearsal  to  enable  realistic  trial  runs  to  be  made.  The  final 
protocols  will  include  the  lessons  learned  and  suggestions  obtained  from  the  dress  rehearsal  participants 
and  observers. 

Pre-  and  Post-  Experiment  Protocols 

A  background  information  questionnaire  will  be  sent  or  taken  to  all  participating  facilities  to  provide  an 
overview  of  the  facility  operation.  This  is  tied  to  the  logistics  to  aid  scheduling  of  the  experiment.  The 
questionnaire  will  include  information  on  the  size  of  facility,  number  of  inspectors,  range  of  aircraft 
inspected,  and  experience  with  Boeing  737s.  This  information  will  be  coded  to  maintain  facility 
confidentiality. 

Procedures  for  storing  all  information  and  its  coding  at  the  end  of  the  experiment  will  be  developed. 
Monitor  Protocols  (Handbook) 

The  protocols  for  the  monitors  will  form  their  Handbook  for  the  experiment,  including  guidance  on  the 
structure  and  purpose  of  the  lap  splice  experiment  and  guidance  on  specific  human-related  issues  that  might 
arise,  such  as: 

•  What  information  may  be  disclosed  to  inspectors  (including  the  objectives  of  the  research  and  the 
ways  the  inspector's  performance  will  be  recorded). 

•  If  and  how  monitors  should  interfere  with  an  inspection  trial  in  progress. 

•  Details  of  maintaining  confidentiality  in  records. 

•  How  the  monitor  should  act  with  other  personnel  and  management  at  the  participating  facility. 

•  How  to  conduct  the  reference  standards  experiment. 

•  What  logbooks,  records  and  data  bases  will  be  maintained. 

These  protocols  arc  defined  in  more  detail  in  Appendix  A  along  with  samples  of  data  gathering  forms. 
Inspectors  Protocols 

These  protocols  will  define  the  procedures  and  data  collection  which  relate  directly  to  the  inspector.  They 
will  address  the  following: 

•  information  to  be  given  to  the  inspector  prior  to  his  participation  in  the  experiment, 

•  the  pre-experiment  briefing  document  to  be  signed  by  the  inspectors  to  indicate  their  agreement  to 
participate, 

•  certain  confidentiality  procedures  specifically  related  to  the  inspector, 

•  inspection  procedures  and  equipment  to  be  used. 
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•  pre-  and  post-inspection  questionnaires. 

In  this  case,  the  Boeing  737  eddy  current  inspection  procedure  used  at  the  facility  will-whenever  possible— 
be  used  by  the  inspector.  Though  the  inspector  will  use  the  equipment  he  would  normally  choose  to  carry 
out  this  inspection,  the  monitor  will  check  the  equipment  to  ensure  it  poses  no  problems  with  the 
experimental  set-up. 

The  protocols  also  include  the  information  to  be  obtained  on  the  inspector  with  respect  to  training, 
experience  with  eddy  currents,  experience  of  Boeing  737  lap  splice  inspection,  and  how  often  these  tasks 
are  normally  carried  out. 


2.6  Logistics 

Logistical  planning  is  the  first  stage  of  implementation  of  an  experiment  as  discussed  in  Section  2.8  of  Ref. 

[1] 

The  logistical  plan  for  the  POD  experiment  requires  several  steps.  These  steps  can  occur  in  both  parallel 
and  serial  order.  Each  of  the  steps  is  discussed  below. 

2.6. 1  Assembly  of  Hardware  and  Dress  Rehearsal 

Procedures  must  be  developed  and  tested  before  the  experiment  begins.  Procedures  for  hardware  assembly 
will  be  completed  after  hardware  fabrication,  and  will  be  practiced  several  times  before  taking  the  hardware 
out  to  a  facility.  This  practice  will  occur  during  a  "dress  rehearsal." 

The  assembly  will  be  set  up  with  the  Foster  Miller  panels  facing  one  direction  and  the  small  specimens 
facing  the  opposite  direction.  This  arrangement  will  provide  both  a  physical  and  a  cognitive  (mental)  break 
between  inspections  of  the  two  types  of  displays.  Although  there  should  not  be  a  long  enough  break  for  the 
inspector  to  change  their  mental  "set"  about  how  they  are  inspecting,  there  should  be  enough  of  a  break  to 
avoid  "jarring"  the  inspector's  willingness  to  treat  the  simulation  as  if  it  were  an  actual  aircraft. 

The  dress  rehearsal  will  embody  the  full  procedures  of  putting  the  hardware  together,  changing  the  order  of 
the  small  specimens  and  data  collection.  The  dress  rehearsal  activities  will  be  held  at  a  third  party 
maintenance  facility  at  the  Albuquerque  International  Airport.  We  expect  that  the  dress  rehearsals  will  be 
observed  by  FAA  personnel,  AANC  personnel,  and  advisory  persons.  The  observers  will  be  asked  to 
evaluate  both  the  system  and  procedures  for  the  research  and  to  provide  suggestions  for  resolving 
problems. 

The  first  portion  of  the  dress  rehearsal  will  consist  of  practice  sessions  in  erecting  and  taking  down  the  test 
specimen  hardware,  from  crated-on-the-truck  and  back  to  that  same  condition.  This  portion  of  the  dress 
rehearsal  will  test  both  hardware  design  and  the  procedures  for  erecting,  making  ready,  and  removing  the 
hardware.  We  expect  that  adjustments  will  be  required  for  the  procedures,  and  perhaps  the  hardware. 
Minor  adjustments  will  be  made  before  the  data  collection  portion  of  the  dress  rehearsal  takes  place. 

The  data  collection  portion  of  the  dress  rehearsal  will  use  eddy  current  technicians  from  a  third  party 
maintenance  facility  and/or  eddy  current  technicians  from  SNL  or  SAIC.  This  portion  must  occur  in 
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several  stages.  In  the  first  stage,  the  inspectors  will  be  given  the  standard  briefing,  followed  by  a 
discussion  about  whether  or  not  the  briefing  materials  are  easy  to  understand  and  accept.  The  next  stage 
will  be  a  tryout  of  the  data  collection  procedures  under  fully  interactive  conditions.  That  is,  both  the 
monitors  and  the  inspectors  will  be  allowed  to  stop  the  procedure  to  ask  questions,  make  observations,  and 
suggest  changes.  After  general  agreement  concerning  the  system  and  procedures,  from  hardware  set-up 
through  data  collection  and  takedown  of  the  hardware,  the  procedures  will  be  followed  exactly.  Necessary 
changes,  at  this  point,  will  be  noted  and  logged,  but  not  made.  This  stage  will  be  repeated  as  many  times  as 
necessary  to  ensure  that  the  procedures  will  yield  a  smooth-running,  repeatable  experiment.  This  stage  will 
be  followed  by  a  general  meeting  of  the  observers,  monitors  and  inspectors  for  final  discussion  of 
procedures.  Thus,  the  dress  rehearsal  could  require  from  as  little  as  five  to  as  much  as  ten  days. 

2.6.2  Scheduling 

There  will  be  more  facilities  scheduled  than  will  be  actually  visited.  Some  of  the  facilities  will  be  chosen  as 
backup  to  others.  Careful  selection  will  allow  minimum  disruption  because  of  unavailability  of  a  facility  at 
the  scheduled  time. 

The  scheduling  program  will  allow  the  monitors  one  week  back  in  their  offices  every  second  or  third  week. 
Since  the  monitors  will  often  observe  two  or  even  three  shifts  to  obtain  the  four  inspectors  needed,  they  will 
be  working  12-  to  16-  hour  days.  Thus,  they  will  need  rest  and  recuperation  time  frequently  or  their  fatigue 
will  reduce  the  accuracy  of  their  observations  and  increase  the  probability  of  uncorrected  errors  in  setting 
up  the  hardware.  The  time  between  trips  will  be  used  profitably  in  catching  up  on  paperwork  and  assisting 
with  data  analysis.  The  early  data  analysis  will  allow  recognition  of  changes  in  the  test  specimens  early 
enough  to  avoid  wasted  effort  in  collecting  unusable  data  because  of  changes. 

2.6.3  Storage  and  Shipment  of  Test  Samples 

The  manager  of  the  experiment  will  control  and  will  be  responsible  for  the  storage  and  shipment  of  the  test 
samples  from  the  time  of  final  characterization.  They  will  arrange  for  secure  storage  facilities  that  will  be 
unchanged  for  the  duration  of  the  experiment.  The  test  samples  will  be  logged  into  and  out  of  those 
facilities,  and  access  to  them  will  be  limited  to  avoid  mishap.  A  custodian  who  has  primary  responsibility 
for  the  test  samples  will  be  designated. 

The  test  samples  will  be  shipped  from  facility  to  facility  using  air  freight  whenever  possible.  When  there 
are  breaks  between  facility  visits  (of  more  than  two  or  three  days),  arrangements  will  be  made  for  safe 
storage,  with  shipment  back  to  central  storage,  if  necessary.  Shipping  arrangements  will  be  the 
responsibility  of  the  custodian  at  the  storage  facility,  although  the  monitors  will  be  responsible  for  oversight 
of  pick  up  and  delivery  on  facility-to-facility  shipments. 

2.6.4  Field  Adjustments  to  Loss  or  Alteration  of  Samples 

A  limited  number  of  test  specimens  (at  least  7)  will  be  manufactured  with  the  experimental  group  of  36 
specimens  and  be  maintained  as  backups.  These  backups  will  be  kept  at  the  major  storage  location  and 
arrangements  made  for  air  shipment  upon  demand. 

If  there  is  an  extreme  change  in  the  expected  pattern  of  inspector  responses  for  a  given  specimen,  then  a 
change  in  crack  characteristics  may  have  occurred.  If  a  change  in  the  specimen  is  suspected,  it  will  be  sent 
back  for  characterization  and  a  new  one  put  in  its  place.  The  new  one  will  not  have  exactly  the 
characteristics  of  the  old,  so  the  changed  pattern  must  be  logged  and  noted  in  the  data  files. 
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In  the  case  of  loss  or  destruction  of  test  specimens,  the  adjustment  will  depend  on  the  percentage  of  the  total 
specimen  group.  Any  losses  less  than  50%  can  possibly  be  adjusted  by  increasing  inspections  of  the 
remaining  specimens.  Some  information  will  be  lost  with  this  procedure;  however,  adjustments  in  analysis 
can  be  made  to  salvage  as  much  information  as  possible.  A  reduced  data  set  is  preferable  to  terminating 
the  whole  experiment. 

2.6.5  Post-Experiment  Archiving  of  Test  Specimens 

When  the  experiment  is  completed,  the  test  samples  will  be  characterized  to  determine  if  any  alterations  in 
the  size  and  distributions  took  place.  (Since  the  test  specimens  are  never  under  tension,  no  changes  are 
anticipated.)  This  characterization  will  be  logged  on  the  test  specimen  inventory  records.  After  that,  the 
specimens  will  be  stored  in  their  shipping  containers  to  form  a  permanent  library  of  well-characterized 
specimens  at  the  AANC  Validation  Center. 
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3.  Implementation 


This  section  describes  the  implementation  efforts  required  to  execute  the  lap  splice  reliability  assessment 
experiment.  In  Ref.[l],  three  major  elements  were  identified:  (1)  Preparation,  (2)  Experiment  Execution, 
and  (3)  Data  Qualification.  The  key  elements  associated  with  each  of  these  activities  were  also  identified. 
In  this  detailed  plan,  that  structure  is  further  expanded  to  specifically  delineate  the  activities  that  must  be 
performed  to  execute  the  experiment.  This  expanded  structure  is  summarized  in  Table  3-1 .  Each  of  the 
major  elements  is  discussed  separately  in  the  sections  that  follow. 

3.1  Preparation 

There  are,  as  noted  in  Table  3-1,  nine  key  elements  associated  with  preparing  these  experiments.  The 
Activities,  as  summarized  in  Table  3-1,  are  discussed  below. 

Monitor  Assignment 

Since  this  experiment  is  designed  to  develop  a  POD  curve  and  to  assure  that  the  human  factors  contributing 
to  that  curve  are  well  understood,  we  propose  that  the  monitors  will  be  a  human  factors  expert  and  an  NDI 
specialist.  (The  performance  of  the  actual  inspection  activities  will  be  done  by  maintenance  center 
personnel  in  each  of  the  centers.)  This  effort  consists  of  selecting  the  monitors,  identifying  alternates,  and 
pursuing  the  process  of  training  and  familiarization  necessary  to  ensure  high-quality,  efficient  operations  at 
the  various  test  sites. 

Site  Coordination 

This  type  of  experiment  requires  that  the  numbers  and  locations  of  sites  be  preselected,  the  timing  of  the 
implementation  of  this  experiment  be  coordinated  with  site  operations,  and  that  alternates  be  available,  if 
for  any  reason  a  primary  site  is  not  available.  These  three  elements  are  the  major  activities  to  be  performed 
under  site  coordination.  The  coordination  process  step  itself  is  an  iterative  one;  from  the  initiation  of 
contact  with  the  upper  management  at  the  potentially  usable  sites,  through  the  process  of  ensuring  near 
term  coordination  just  prior  to  the  arrival  of  the  equipment  and  personnel  at  the  test  site.  This  activity  is 
vital  to  ensuring  a  smooth,  efficient,  and  unbiased  experiment  sequence. 

Safety 

Safety  is  always  a  significant  consideration  in  all  test  efforts.  The  activities  shown  in  Table  3-1  address  the 
review  of  OSHA  requirements  to  ensure  that  all  basic  safety  rules  are  met  in  the  implementation  of  the 
experiment.  To  this  end,  a  preliminary  hazards  analysis  that  will  identify  potentially  hazardous  activities 
will  be  performed.  The  hazard  mitigation  approaches  that  are  be  taken  will  be  identified  to  assure  the 
safety  of  these  operations.  Each  site  will  have,  in  compliance  with  good  working  conditions,  a  safety 
program  of  its  own.  It  is  most  important  that  when  implementing  this  experiment,  these  site  unique  safety 
considerations  be  examined  so  that  all  of  the  activities  performed  by  the  experiment  team  comply  with  site 
safety  rules.  The  dress  rehearsal  will  help  define  areas  where  safety  issues  arc  likely  to  arise. 

Sample  Acquisition 

In  preparing  for  th  ;  experiment,  sample  acquisition  goes  beyond  receipt  of  the  panels  and  the  specimens.  A 
receiving  inspection  will  be  performed  to  ensure  that  the  panels  have  been  prepared  in  accordance  with  the 
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drawings  and  specifications.  Each  panel  needs  a  carefully  marked,  unique  identification  to  ensure  proper 
set-up  at  each  site.  The  generic  experiment  plan  addresses  the  need  for  and  the  process  of  providing 
pre-experiment  characterization  of  the  panels,  cracks,  etc.  This  is,  of  course,  a  vital  part  of  preparing  for 
the  experiments. 

Test  Equipment  Acquisition 

All  the  equipment  (meters,  tools,  gages,  etc.)  required  for  the  set-up,  tear  down,  and  check  out  of  the 
experiment  at  each  site  will  be  received,  checked  out,  and  calibrated,  if  necessary. 

Shipping  and  Handling 

Since  the  test  is  being  performed  at  multiple  sites,  shil  o  ig  and  handling  is  an  important  part  of  the 
preparation  activities.  Containers  will  be  fab.  ‘ed.  upon  receipt  of  the  containers,  they  will  be  inspected 
for  compliance  with  drawings  and  specificatio.  Transportation  arranged  for  the  samples  and  the  handling 
equipment  will  be  finalized  during  the  preparation  phase.  Handling  equipment,  as  it  is  received,  will  be 
inspected  for  compliance  with  drawings  and  specifications.  Further,  the  assembly  and  tear  down  of 
handling  equipment,  using  the  tools  and  the  procedures  that  will  be  provided  at  the  site,  will  be  checked  out 
as  a  part  of  the  shipping  and  handling  activities. 

Experiment  Integration 

To  ensure  a  smooth  and  efficient  set  of  operations  at  each  site,  experiment  integration  is  a  major 
preparation  effort.  Each  piece  of  the  experiment,  that  in  some  sense  must  be  made  compatible  with  other 
pieces,  will  be  checked  out  during  these  efforts.  As  noted  in  Table  3-1,  integration  of  specimens  and 
containers,  specimens  and  mounting  fixtures,  will  be  performed  to  ensure  that  all  of  the  hardware  elements 
fit  together  properly.  Additionally,  as  with  any  new  procedure,  trial  runs  of  protocols  and  use  of  the 
documentation  (i.e.,  detailed  test  plan)  will  be  performed  to  ensure  that  all  significant  activities  are  properly 
structured  in  the  documentation,  and  further  to  ensure  that  all  equipment  is  functioning  as  anticipated  in  the 
implementation  of  the  protocols.  A  baseline  set  of  data  on  the  specimens  is  a  vital  ingredient  to  the  overall 
evaluation  process.  These  carefully  taken  baseline  data  will  be  acquired  as  a  part  of  the  experiment 
integration  efforts. 

Dress  Rehearsal 

The  dress  rehearsal  will,  to  the  maximum  extent  possible,  replicate  the  "real  life"  experiment  as  it  will  be 
performed  at  each  of  the  sites.  The  monitors  (human  factors  expert  and  NDI  technologist)  will  go  through 
the  entire  protocol  with  the  dress  rehearsal  inspectors  as  if  this  were  an  on-site  activity.  In  addition, 
observers  will  be  invited  to  critique  the  process.  To  ensure  that  the  observers  are  aware  of  the  overall 
structure,  objectives,  basic  approach,  etc.,  relevant  to  these  experiments,  a  briefing  for  the  observers  will  be 
prepared  and  presented.  Finally,  at  the  conclusion  of  the  dress  rehearsal,  the  observers  will  provide  their 
constructive  comments  and  observations  through  a  debriefing  to  the  experiment  team. 

Storage 

We  anticipate  that  there  will  be  times  when  the  specimens  and  the  handling  equipment  will  have  to  be 
stored  between  use  at  test  sites.  This  element  of  the  preparation  phase  addresses  storage  areas,  storage 
containers,  storage  conditions,  etc.,  so  that  a  smooth  transition  from  the  completion  of  an  experiment  at  a 
particular  site  into  the  storage  area  and,  subsequently  out  of  the  storage  area  to  the  next  test  site  can  be 
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effected  as  necessary.  Additionally,  storage  of  all  data  will  be  addressed  in  this  phase  of  the  preparation 
activities.  A  central  storage  area,  as  well  as  a  uniform  process  for  inputting  the  data  to  the  central  data 
base,  will  be  set  up  and  checked  out  as  a  part  of  this  effort. 


3.2  Experiment  Execution 

Table  3-1  lists  five  key  elements  under  experiment  execution.  The  structure  set  up  in  Table  3-1  for  the  key 
elements  lays  out  the  sequence  of  activities  and  events  that  constitute  the  performance  of  the  experiment 
itself.  Each  of  the  five  elements  is  discussed  below 

Site  Introduction 

To  ensure  that  all  parties  who  are  a  part  of  the  experiment  efforts  are  aware  of  the  objectives  and  the 
general  sequence  of  events  that  comprise  the  experiment,  an  introductory  briefing  will  be  prepared  and 
presented  as  the  first  activity  at  each  site.  At  this  time,  any  questions  that  management  or  personnel  at  the 
maintenance  facility  have  concerning  this  experiment  or  its  implementation  will  be  addressed. 

Site  Preparation 

Site  preparation  consists  of  all  of  the  activities  necessary  from  the  time  of  arrival  of  the  equipment  and 
personnel  to  the  time  when  the  maintenance  facility's  inspectors  are  ready  to  initiate  the  experiment.  As 
noted  in  Table  3-1,  the  major  activities  consist  of  receiving  the  equipment,  assembling  of  the  mounting 
fixture,  mounting  specimens  on  the  mounting  fixture,  and  performing  any  necessary  pre-test 
instrumentation  calibration.  All  of  the  activities  performed  in  the  site  preparation  area  will  be  performed 
by  the  two  monitors. 

Experiment  Performer's  Briefing 

At  each  site,  the  maintenance  facility's  inspectors  participating  in  the  experiment  will  be  given  an  overview 
briefing.  This  briefing  will  outline  the  experiment,  review  the  protocol,  and  ensure  that  the  inspector 
understands  what  he/she  has  to  do  to  implement  the  experiment.  In  addition,  there  will  be  a  pre-test 
interview/discussion.  Some  of  the  human  factors  related  to  the  performing  of  this  experiment  clearly 
involve  experience,  training,  etc.,  for  each  inspector.  This  type  of  information  will  be  gleaned  during  the 
pre-test  interview. 

Experiment  Implementation 

The  actual  experiment  will  be  performed  at  this  point  in  the  sequence.  Prior  to  inspections  of  the  test 
samples,  the  NDI  monitor  will  characterize  the  reference  standards  used  in  each  facility  against  a  master 
calibration  block.  The  recording  of  conditions  as  noted  in  Table  3-1  will  be  done  by  the  monitors. 
Equipment  identification  recording  will  be  a  joint  activity  between  the  inspectors  and  the  monitors.  The 
inspection  sequence  on  the  test  samples  will  be  performed  by  the  inspection  team  in  accordance  with  the 
site's  basic  inspection  procedures,  and  using  the  site's  inspection  tools  and  test  equipment.  A  major  role  of 
the  monitors  is  to  observe  this  process  and  to  highlight  significant  activities  that  are  related  to  the  human 
factors  elements.  These  factors,  in  the  performance  of  the  inspections,  will  be  recorded  in  accordance  with 
the  observation  protocol.  For  each  site  multiple  inspections  will  be  performed  by  different  inspectors  (with 
one  repeat  trial  -  same  inspector).  Recording  conditions,  identifying  equipment,  and  recording 
observations  will  be  done  for  each  of  the  replications  of  the  experiment. 
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Post  Experiment  Efforts 


When  the  experiments  are  completed,  the  set  of  activities  shown  in  Table  3-1  will  be  performed.  The  first 
item.  Post  Experiment  Interviews  of  Performers,  provides  the  final  sets  of  "data"  related  to  the  experiments. 
The  rest  of  the  activities  listed  under  post  experiment  efforts  are  directed  towards  the  tear  down  of  the 
experimental  set-up  and  the  preparation  for  moving  the  experimental  set-up  to  the  next  site.  The  final  item. 
Debrief  Site  Management,  is  designed  to  ensure  that  good  communications  are  maintained  with  the 
maintenance  site  management  personnel  concerning  the  experiments  and  the  interaction  with  the  inspection 
crew. 

3.3  Data  Qualification 

A  significant  element  in  all  well-run  experiments  is  the  implementation  of  a  process  to  ensure  that  all  of  the 
required  data  has  been  acquired,  that  all  of  the  data  are  "good"  data,  and  that  all  data  are  properly  identified 
and  marked.  The  activities  shown  in  Table  3-1,  under  Data  Qualification,  address  these  areas. 

The  data  sets  will  be  marked  at  the  completion  of  each  experimental  run.  The  observation  information 
compiled  in  accordance  with  protocol  will  be  prepared  subsequent  to  the  completion  of  each  run,  and  will 
be  carefully  identified  as  to  date,  inspector,  and  shift  to  ensure  proper  correlation.  At  the  completion  of 
each  run,  the  set  of  raw  data  will  be  examined  to  ensure  that  all  required  sets  of  information  have  been 
taken.  A  significant  part  of  this  activity  is  to  review  all  of  the  forms  required  by  the  protocol,  to  ensure  that 
all  elements  of  data  required  by  the  forms  have  been  addressed  and  thoroughly  documented.  If  the  records 
are  handwritten,  legibility  is  a  key  issue.  (The  current  approach  anticipates  using  notebook  computers  with 
preprogrammed  forms  to  ensure  legibility  and  easy  translation  of  the  data  into  a  summary  data  base.) 

Finally,  a  quick  review  by  the  human  factors  expert  and  the  NDI  specialist  immediately  after  the  run  to 
ensure  that  no  anomalous  or  obviously  inconsistent  sets  of  results  are  contained  in  the  data  is  worthwhile. 
Misunderstandings  can  be  cleared  up  immediately  if  the  test  set-up  and  the  people  who  are  performing  the 
inspections  are  still  in  place.  Similarly,  an  improperly  implemented  element  of  the  protocol  can  be  dealt 
with  while  the  test  set-up  is  still  in  place. 

The  data  qualification  activity  is  not  designed  to  analyze  the  results  or  to  attempt  to  draw  conclusions 
concerning  the  interpretation  of  the  data.  Rather,  it  is  designed  to  ensure  that  all  of  the  data  needed  by  the 
analysts  are  clearly,  correctly,  and  accurately  accumulated  in  the  process  of  performing  the  experiment. 
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Table  3-1.  Experiment  implementation  elements. 


Major 

Element 

Key  Factors 

Activities 

Preparation 

Monitor  Assignment 

•  Selection 

•  Alternates 

•  Training/Familiarization 

Site  Coordination 

•  Final  Selection 

•  Scheduling 

•  Alternate  Site  Identification 

Safet; 

•  Review  of  OSHA  Requirements 

•  Preliminary  Hazards  Analysis 

•  Cosideration  of  Site  Unique  Safety 
Elements 

Sample  Acquisition 

•  Receipt 

•  Identification 

•  Pre-experiment  Characterization 

Test  Equipment  Acquisition 

•  Receipt 

•  Calibration 

•  Checkout 

Shipping  and  Handling 

•  Containers 

•  Transportation 

•  Handling  Equipment 

Experiment  Integration 

•  Integration  of  Samples  and  Containers 

•  Integration  of  Samples  and  Frame 

•  Trial  Runs  of  Protocols  and 
Documentation 

•  Baseline  Data  Acquisition 

Dress  Rehearsal 

•  Briefing  Observers 

•  Experiment  Execution 

•  Debriefing  by  Observers 

Storage 

•  Samples 

•  Frame  (Mounting  Fixture) 

•  Test  Equipment 

•  Data 

Execution 

Site  Introduction 

•  Briefing 

Site  Preparation 

•  Receipt  of  Equipment 

•  Movement  to  Experiment  Area 

•  Unpacking 

•  Inspection 

•  Frame  Assembly 

•  Sample  Mounting 

•  Pretest  Instrument  Calibration 

•  Gage  Block  "Calibration" 

•  Monitoring  Instrument  Checkout 
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Table  3-1.  Experiment  implementation  elements  (cont.) 


Inspectors  Briefing  •  Overview  Briefing 

_ •  Pretest  Interview/Discussion _ 

Experiment  Implementation  •  Reference  Standards  Experiment 

•  Recording  of  Condition 

•  Temperature 

•  Humidity 

•  Lighting 

•  Noise 

•  General  Operating  Conditions 

•  Equipment  Identification  Recording 

•  Make  and  Model 

•  Serial  Number 

•  Condition  (calibration,  etc.) 

•  Perform  Inspection  Sequence  on 
Samples 

•  Record  Observations  per  Protocol 

•  Repeat  Inspection  Sequence  as 

_ Required  by  Protocols _ 

Post  Experiment  Efforts  •  Post  Experiment  Interview  of 

Inspectors 

•  Disasembly  of  Frame/Sample 
Assembly 

•  Inspection  of  Specimens 

•  Refurbishment  as  Required 

•  Pack  and  Prepare  for  Shipment 

•  Ship  to  Next  Site 

•  Aggregate  and  Ship  Data 

_ _ »  Debrief  Site  Management _ 

Data  Identification  of  all  Data  Sets  •  Marking 

Qualification 

•  Correlation  (Data,  Observations, 

_ Records,  etc.) _ 

Complete  Data  Set  Determination  •  All  Raw  Data  Taken 

•  All  Forms  Completed  as  Required 
by  Protocol 

•  All  Entries  Completed  as  Required 
by  Protocol 

_ •  All  Data  Legible _ 

Quality  Check  •  Results  Reasonable  (Quick  Look 

_ Evaluation) _ 


4.  Expected  Results  and  Data  Analysis 

An  outline  of  the  planned  data  analysis  was  given  in  Section  2.4.  This  section  discusses  in  more  detail  how 
the  data  are  to  be  stored,  made  available,  and  presented.  The  types  of  analysis  needed  can  be  performed  by 
various  commerc'ally  available  software  packages.  Specifically  he  SAS  software  for  statistical  analysis 
is  available. 

In  a  reliability  assessment  program  that  entails  visiting  many  facilities,  it  is  important  that  data  analysis 
activities  be  done  in  parallel  with  gathering  the  data.  Inspection  results  will  be  taken  from  the  tape  on 
which  they  are  recorded  and  will  be  transferred  to  an  electronic  data  base.  The  transfer  to  an  electronic 
data  base  will  be  done  in  timely  manner  so  that  preliminary’  data  analysis  can  take  place.  This  preliminary 
data  analysis  can  give  indications  of  problems.  For  example,  an  unflawcd  site  consistently  being  flagged  as 
a  flaw  could  indicate  changes  in  the  specimen. 

The  transcription  of  hard  copy  inspection  results  to  the  electronic  data  base  will  likely  occur  in  the  field. 
Both  the  electronic  data  base  and  the  hard  copy  will  then  be  sent  (under  separate  cover)  to  Sandia  where 
preliminary  analysis  w  ill  be  done.  On-site  analysis  of  inspection  data  by  the  personnel  who  will  be 
monitoring  the  experiment  will  not  be  done.  This  is  to  assure  that  flaw  information  is  not  available  at  the 
inspection  sites. 

For  summary’  presentations,  individual  POD  curves  for  each  inspector  will  be  shown  on  one  plot.  The  total 
variation  due  to  inspectors  will  be  visually  apparent.  Similarly,  POD  curves  will  be  fit  to  the  inspection 
results  averaged  across  inspectors  within  a  facility.  A  plot  will  be  made  with  individual  curves  representing 
each  facility.  The  result  is  a  visual  presentation  of  facility  variation. 

Plots  showing  inspector-to-inspector  and  facility-to-facility  variation  are  fundamental.  However,  the 
significance  of  the  observed  variation  depends  on  factors  of  the  experimental  design,  including  the  number 
of  flaws  inspected  and  the  number  of  inspections.  The  statistical  significance  will  be  assessed  and  reported 

Individual  ROC  curves  will  also  be  displayed  on  the  same  plot.  Similarly,  aggregated  ROC  curves  by 
facility  will  also  be  put  on  a  single  plot.  Thus,  the  same  variation  sources  as  discussed  for  the  POD 
analysis  will  be  displayed  in  the  ROC  presentations.  In  addition,  the  ROC  curves  derived  when  constrained 
to  specified  ranges  of  crack  length  w  ill  be  constructed  and  displayed  on  the  same  plot.  This  will  give  a 
visual  presentation  of  the  relationship  between  the  ROC  curve  and  crack  length. 

The  results  of  ANOVA  and  MANOVA  analyses  on  the  a's  and  P's  from  individual  POD  fits  and 
interpretations  of  the  ROC  measures,  such  as  the  "A"  index,  will  also  be  part  of  the  basic  reporting.  The 
primary  explanatory  variables  included  in  these  analyses  will  be  those  that  were  controlled.  However, 
exploratory  analysis  will  be  pursued  using  various  observed  conditions  as  possible  explanatory  factors. 

An  experiment,  such  as  this  one,  that  obtains  data  from  many  facilities,  industry  wide,  will  generate  much 
interest.  It  is  important  to  make  the  full  dataset  of  inspection  results  available  for  general  distril  .tion 
This  will  result  in  independent  analyses  and  w  ill  facilitate  open  discussion  of  the  results.  Therefore,  a  data 
base  description  document  will  be  prepared.  A  PC-based  flat  ASCII  file  will  be  made  available  along  with 
the  data  base  documentation  for  those  requesting  the  data. 
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5.  Preliminary  Action  Plan 


The  main  efforts  of  any  reliability  assessment  experiment  are  spent  in  planning,  implementing,  analyzing, 
determining  POD  estimates,  and  producing  the  final  report.  The  process,  however,  is  not  complete  until  the 
initial  goals  of  the  program  have  been  reviewed  and  compared  with  the  program  results.  Normally,  this  is 
accomplished  by  making  a  list  of  recommendations,  reaching  certain  conclusions,  and  recommending  what 
elements  require  furuier  investigation. 

During  the  course  of  conducting  the  reliability  assessment  experiment,  a  great  deal  of  information  will  be 
collected  and  analyzed.  Based  on  the  data  collected,  it  is  not  unlikely  that  certain  courses  of  action  can  be 
recommended  that  could  have  an  impact  on  the  overall  inspection  process  and  improve  its  reliability.  A 
Preliminary  Action  Plan  will  be  determined,  based  on  the  lessons  learned  and  the  data  collected  while 
conducting  the  experiment. 

This  plan  will  consider,  but  not  be  limited  to,  actions  pertaining  to  such  inspection  aspects  as: 

•  Standardized  calibration  techniques  for  equipment,  sensors,  and  calibration  blocks 

•  Inspector  training  programs 

•  Standardized  certification  examinations 

•  Rc-cxaminations 

•  Improvements  in  environmental  conditions 

•  Improvements  in  staging 

•  Improvements  in  fixtures 

•  More  meaningful  data  collection 

Based  on  the  information  available,  methods  for  implementing  identified  concerns  will  be  determined. 

The  benefits  of  action  based  on  reliable  inspection  data  are  manifold.  They  should  lead  to  increased 
confidence  in  inspection,  giving  greater  passenger  confidence.  By  identifying  the  key  reliability  issues, 
maintenance  procedures  can  be  optimized  to  improve  performance  at  minimum  cost  and  inspection  staff 
can  gain  a  greater  understanding  of  their  own  jobs.  The  result  will  be  a  more  productive  maintenance 
organization  with  inspectors  having  the  necessary  resources  and  support  to  better  perform  their  job. 
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APPENDIX  A 


PROTOCOLS 


a-; 


LIST  OF  FIGURES 

A-l 

Facility  characteristics  checklist 

A-5 

A-2 

Data  recording  form 

A-7 

A-3 

Outbriefing  questions  for  inspectors 

A-8 

A-2 


This  appendix  gives  more  information  on  the  requirements  for  the  monitor's  protocols  and  examples  of 
draft  data  sheets  and  briefing  materials.  These  protocols  will  be  completed  before  the  dress  rehearsal  and 
modified  if  necessary  in  light  of  the  performance  during  those  trials. 

Experimental  Plan 

This  document  will  present  the  overall  program  and  its  objectives  and  methodology.  It  will  also  include  the 
detailed  schedule,  hardware  descriptions  and  identification  system  (for  panels  and  specimens),  and  other 
pertinent  information  from  the  implementation  section  of  this  document. 

Start-Up  Procedures 

A  checklist  of  the  hardware,  materials,  inspection  procedures,  log  books,  experimental  orders  and  plan  for 
the  day. 

Panel  Layout  Diagrams 

These  diagrams  will  provide  the  designs  for  specimen  placement  for  each  research  run.  The  specimens  and 
panels  will  be  identified  by  serial  number  and  location,  in  order,  from  a  starting  end  for  each  lap  splice. 

Briefing 

This  document  will  present  the  standard  introduction  given  to  the  inspectors  including  an  explanation  of  the 
research,  confidentiality  safeguards,  and  agreement  to  participate.  A  separate  briefing  will  be  given  to  the 
facility  management. 

Reference  Standards  Experiment 

This  will  detail  the  procedure  for  carrying  out,  by  the  monitor,  the  reference  standards  experiment.  It  will 
be  based  on  the  calibration  section  of  the  Boeing  737  inspection  procedure. 

Pre-Trial  Question  List 

This  checklist  will  help  the  monitors  gather  necessary  background  information  and  demographic 
information  about  the  inspectors  before  they  begin  the  trial. 

Inspector  Supervision  Procedures 

This  document  will  describe  the  aim  and  requirements  of  the  inspectors  from  the  monitor's  point  of  view.  It 
will  provide  guidance  to  the  monitors  on  how  to  proceed  under  particular  circumstances. 

Trial  Checklist 

This  checklist  will  give  the  actions  to  be  taken  by  the  monitor  during  the  inspections  (e  g.,  filling  in 
questionnaires  and  recording  relevant  data  defined  above).  A  summary  of  environmental  factors  and 
inspection  data  that  arc  to  be  recorded  is  shown  in  the  Facilities  Characteristics  Checklist,  attached. 
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Data  Recording/Transfer  Procedures 

The  procedure  for  transferring  records  of  the  inspector's  work  will  be  provided  in  this  document.  An 
outline  is  given  on  the  Data  Recording  Form,  attached.  Current  recommendations  are  that  the  inspector 
marks  his  findings  on  the  specimens  and  panels  and  that  these  are  then  transferred  by  the  monitor  to  the 
data  base.  The  checklists  will  be  recorded  directly  to  the  data  base.  A  check  procedure  for  transfer 
accuracy  will  be  included. 

Close-Down 

This  document  will  describe  the  methods  for  filing  paperwork,  computer  disks,  etc.,  at  the  end  of  a  session. 
It  will  also  provide  steps  for  dismantling,  storing,  and  shipping  the  hardware.  A  checklist  will  be  provided 
for  ensuring  that  all  equipment,  data,  and  forms  are  properly  packed  and  labeled. 

End  of  Trial  Debriefing 

This  is  the  post-experiment  questionnaire  which  the  monitor  will  complete  in  consultation  with  the  inspector 
in  order  to  determine  both  the  inspector's  attitudes  to  and  opinions  of  the  trial  and  how  he/she  is  feeling.  A 
draft  structure  of  the  interview  is  given  on  the  Outbricfing  Questions  for  Inspectors  form,  attached.  A 
separate  debriefing  will  be  given  to  the  management  to  thank  them  and  give  general  overview  of  how  the 
trials  proceeded.  No  specific  inspection  or  personnel  data  will  be  disclosed. 
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Hangar  Size 


LIGHTING  - 


TEMPERATURE  - 


Artificial _  Outdoors _  _  Cold  &  Drafty 

_  Glaring  _  Cold 

_  Bright  _  Comfortable 

_  Average  _ Hot 

_  Dim  _  Very  Hot 

_ _  Shadowed  _  Hot  &  Humid 


Light  meter  reading  (foot-candles)  at  time  of  inspection 


GENERAL  ATMOSPHERE  - 


MANAGEMENT  - 


Gloomy 

High-pressured 

Noisy  (see  below) 

Cheerful 

Quiet 

Relaxed 


High  pressure?  (describe) 
Schedule  constraints?  (describe) 
Describe  intercommunications 
Team  cohesion?  (describe) 
Distracting  tasks?  (describe) 


TOOLS  - 


HOUSEKEEPING  - 


Availability?  (describe)  _  Clean? 

Employee  furnished?  (describe)  _ Neat? 

Adequate?  (describe)  _  Well  organized? 

Calibrated?  (describe)  _ Describe  unusual  characteristics  (unusually  good 

or  bad) 


INSPECTOR  - 


NOISE  - 


Physical  condition  (describe) 
Attentiveness  (describe) 

Age 

Sex 


Steady  level 
Intermittent 
High 
Low 

Noise  meter  reading  (dB) 


Figure  A-l .  Facility  Characteristics  Checklist 
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Eddy  Current  Data  Sheet 
Monitor  collects  with  each  inspection. 


Date: _ 

Inspector  ID: 

Layout  for  design _ 

System  Operating  Ambient  Temperature _ 

State  other  equipment  environmental  constraints _ 


Test  Frequency  _  Scan  Speed _  Filtering 

Horizontal  Gain _  Vertical  Gain _  Lift-off  technique 

Coil  Output  Impedance 


Others 

Others 


Scanning  Technique _  Digitization  _ 

Calibration  Level  _  Inspection  Threshold 

Calibration  on  paint/unpainted _ (tape) 


Probe 

Contact _ 

Differential 

Pancake 


Coil  Diameter 


Noncontact 

Absolute _ 

Toroid  Coil 
Shielding  _ 


Attach  a  sketch  of  the  inspection  set-up.  Include  part  orientation  with  respect  to  flaw  orientation  and  eddy 
current  direction. 


Describe  technique  for  analyzing,  rejecting  and  recording  a  defect  signal  or  affix  procedure  used. 


Describe  deviations  from  procedure. 


Figure  A- 1.  Facility  Characteristics  Checklist  (cont.) 
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DESIGN 


The  form  will  have  heading  spaces  and  three  columns  for  each  inspector.  The  following  will  be  recorded  in 
the  heading  spaces 

•  Code  Designation 

•  Date 

•  Start  Time 

•  End  Time 

•  Equipment  used  (type,  cables,  probes,  calibration  standard)  [see  the  Facilities 
Characteristics  Checklist  for  eddy  current  specific  form] 

The  columns  will  be: 

•  Crack  response  (Y /N)  [this  data  transferred  from 

•  Confidence  rating  fuselage] 

(1  =  very  sure,  2  =  pretty  sure, 

3  =  some  justification) 

•  Comments:  It  is  very  important  to  record  break  start  and  stop  times  and  any  other 
information  that  would  help  in  interpreting  the  data  later  on,  for  example,  repeated 
inspections. 


Figure  A-2.  Data  Recording  Form 
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NOTE.  READ  ESSENTIALLY  AS  WRITTEN,  AFTER  EXPLAINING  THAT  WE  NEED  SOME 
INFORMATION  THAT  WE  COULD  USE  TO  IMPROVE  THIS  EXPERIMENT 


1 .  How  much  was  working  on  the  test  piece  like  working  on  an  aircraft? 

_  Very 

_  Somewhat 

_  Similar  in  some  ways,  not  in  others 

_ _  Somewhat  dissimilar 

_  Very  dissimilar 

Not  at  all  similar 


2.  What  are  the  major  differences  that  you  feel  arc  important?  (Pick  the  three  most  major.) 

a. 

b. 

c. 

3.  How  would  you  suggest  that  we  improve  this  research? 

4.  How  much  did  the  watcher  bother  you? 

_  A  lot 

_  Some 

Not  at  all 


5.  How  often  have  you  done  this  job  of  crack  detection? 

6.  How  long  has  it  been  since  you  last  did  this  job  of  crack  detection? 

7.  How  much  training  have  you  had  on  this  equipment? 

8.  What  do  you  think  the  priority  of  NDI  is  compared  to  the  other  things  that  you  need  to  do? 

•  To  you  personally: 

•  To  your  management: 


Figure  A-3.  Outbricfing  Questions  for  Inspectors 
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APPENDIX  B 


SPECIMEN  DESIGN  AND  MANUFACTURE 


LIST  OF  FIGURES 


B-l  Fatigue  loading  for  crack  propagation  in  upper  plate  B-3 

B-2  Tensile  test  setup  used  to  produce  off  angle  cracks  B-4 

B-3  Elimination  of  crack  starter  notch  B-4 

B-4  Demonstration  plates  1  and  2  B-5 

B-5  Full-scale  test  panel  built  by  Foster-Miller  B-7 

B-6  Full-scale  panel  test  facility  B-8 

B-7  Panel  test  frame  structure  B-8 

B-8  Panel  installation  in  test  fixture  B-9 

B-9  Substructure  of  support  frame  for  small  test  specimen  B-9 

B-10  Assembly  of  five-foot  test  section  with  test  specimen  and  end  suppoprts  B-10 

B- 1 1  Variation  in  end  support  assembly  to  adjust  height  and  angle  of  inspection  surface  B-l  1 

B- 1 2  Experiment  test  setup  B- 1 1 

B-l 3  Test  specimen  mounting  to  mock  fuselage  frame  B-l 2 
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A  brief  description  of  the  generation  of  the  flaws  as  well  as  the  frame  design  is  given  in  this  appendix. 

Small  Specimens 

The  cracked  upper  plates  will  be  produced  as  follows: 

1 .  An  oversized  "upper  plate"  will  be  placed  in  a  mechanical  test  machine  (Figure  B-l).  Undersized  holes 
will  be  pre-drilled  at  the  locations  where  the  cracks  are  to  be  propagated.  Since  fatigue  cracks  normally 
grow  in  the  direction  perpendicular  to  the  load,  off-angle  cracks  will  be  produced  by  setting  up  the 
tensile  test  in  the  manner  depicted  in  Figure  B-2.  Increasing  either  the  width  or  height  of  the  panel 
specimen  will  greatly  increase  the  plate  size  required  to  generate  off-angle  cracks. 

2.  Crack  starters  will  be  sequentially  placed  in  these  holes  -  using  crack  growth  information  for  2024-T3 
aluminum  -  to  achieve  the  desired  crack  distribution.  Any  means  that  is  used  to  initiate  the  cracks  will  be 
eliminated  from  the  final  specimen.  Figure  B-3  shows  how  this  is  accomplished  where  a  crack  starter 
notch  is  removed  when  the  final  5/32-in.  diameter  hole  is  drilled. 

3.  Sample  plate  flaw  specifications  are  shown  in  Figure  B-4.  Although  exact  crack  lengths  are  given, 
limitations  in  fatigue  growth  will  not  allow  all  of  the  cracks  to  have  the  desired  lengths.  Rather,  the 
cracks  will  be  sequentially  initiated  in  an  attempt  to  produce  the  desired  length  distribution.  The  longest 
crack  on  each  particular  plate  will  be  used  as  the  controlling  factor.  When  it  has  been  reached,  the 
loading  will  cease  and  all  of  the  existing  cracks  will  be  me  asured.  At  that  point,  it  will  be  determined  if 
continued  loading  will  produce  a  better  crack  distribution.  This  iterative  process  will  be  carried  out 
during  the  production  of  each  cracked  upper  plate. 


B-3 


Figure  B-2.  Tensile  test  setup  used  to  produce  off  angle  cracks. 
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da/dN  information,  to  achieve  desired 
length  distribution 


Figure  B-3.  Elimination  of  crack  starter  notch. 
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After  the  cracked  upper  plates  have  been  produced,  this  plate  will  be  riveted  into  the  final  panel  specimen 
shown  in  Figure  B-i.  This  task  involves  drilling  all  remaining  rivet  holes  and  associated  counterbores.  This 
assembly  procedure  will  adhere  to  aviation  specifications  in  general  and  to  the  Boeing  specifications  where 
applicable. 

Proof-of-Concept  Demonstration  Plates.  To  deliver  a  tangible  proof-of-concept,  the  AANC  had  two  sets  of 
demonstration  plates  fabricated.  Two  candidate  companies  were  chosen  to  produce  these  demonstration 
plates  Figure  B-4  shows  the  crack  distribution  specifications  of  the  two  test  specimen  upper  plates.  Both 
companies  were  able  to  do  a  reasonable  job  of  meeting  the  specifications. 


1.  All  hole  location  tolerances  are  +0.003"  in  both  directions 

2.  All  crack  length  tolerances  are  +  0.005";  cracks  emanate  from  right  side,  left 
side,  or  both  sides  as  shown  in  drawing 

3  Final  hole  diameter  .s  5/32“  ±  0.002“ 

A.  All  cracks  are  along  hole  line  (angle  =  0°) 

DEMONSTRATION  PLATE  #1 


t  All  hole  location  tolerances  are  +  0.003"  in  both  directions 

2  All  crack  length  tolerances  are  ♦  0.005/  cracks  emanate  from  right  side,  left 
side,  or  both  sides  as  shown  in  drawmn 

3  Final  hole  diameter  is  5/32"  ±  0  002“ 

4  All  cracks  are  o*f  ang’e  (angle  =  22°) 

DEMONSTRATION  PLATE  *2 


Figure  B-4.  Demonstration  Plates  1  and  2.  Hole  locations  arc  ±  .003  inch.  All  cracks  on  plate  1  are 
horizontal.  All  cracks  on  plate  2  arc  off  angle  (22°  from  horizontal). 
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Aircraft  Test  Panel 


Figure  B-5  shows  a  mechanical  drawing  of  the  larger  panel.  The  planform  dimensions  which  include  the 
connection  flanges  for  the  load  frame,  are  10  ft  long  by  6  ft  high.  The  stringer  spacing  is  an  adjustable 
variable  and  panels  produced  to  date  have  had  either  8-in.  or  9.6-in.  stringer  spacing.  The  splice  is  joined 
by  three  longitudinal  rivet  rows;  the  center  rivet  row  also  attaches  the  skin  to  a  stringer.  One-inch  wide  tear 
straps  arc  positioned  in  the  hoop  direction  on  20  inch  centers  at  each  frame.  The  frames  arc  attached  to  the 
skin  in  each  stringer  bay  by  a  fabricated  angle  shear-clip.  All  through-skin  rivets  are  5/32-in.  diameter,  low 
profile,  shear  head  100-degree  countersink  rivets.  Photographs  of  typical  panels  are  shown  in  Figure  2-2. 
The  panels  arc  fabricated  by  East  Coast  Aero  Tech,  Hanscom  Field,  MA,  under  the  supervision  of 
Foster-Miller. 

Load  Frame  Design.  This  load  frame  applies  simultaneous  internal  pressure  (hoop  load)  and  longitudinal 
loads  to  the  curved  segment  of  stiffened  aircraft  panel.  Figures  B-6  and  B-7  are  photographs  of  the  entire 
test  facility.  This  test  fixture  was  designed  to  include  the  following  features: 

1 .  Fixture  to  panel  connections  reproduce  conditions  of  full-circle  fuselage. 

2.  Panel  loadings  simulate  normal  pressurization  loads  in  the  panel.  The  load  is  proportioned  to  produce 
an  axial  stress  of  half  of  the  hoop  stress  because  of  the  internal  pressure,  as  it  exists  in  the  full-scale 
round  aircraft  fuselage;  proper  phasing  of  these  loads  is  also  maintained 

3.  Providing  cyclic  pressurization  loads. 

4.  Loading  the  panel  to  a  pressure  equivalent  to  one-half  the  panel’s  ultimate  capacity. 

The  general  approach  to  panel  testing  (i.e.,  generating  flaws  in  the  panels)  is  to  anchor  two  long  sides  and 
one  end  of  the  panel  to  the  fixture.  Panel  loading  is  achieved  by  pulling  on  one  curved  end  and  pressure 
loading  the  panel  on  its  concave  side  by  pressurized  water.  The  pressure  load  is  contained  by  the  fixture, 
by  an  inflatable  seal  between  the  fixture  and  the  panel,  and  by  the  panel.  Figure  B-8  is  a  schematic  of  the 
panel  mounted  in  the  test  fixture. 


B-7 


Figure  B-5.  Full-scale  test  panel  built  by  Foster-Miller. 


Figure  B-8.  Panel  installation  in  test  fixture. 


Test  Frame/Support  Structure 

Test  Frame  Sections.  For  shipping  and  handling  purposes,  the  test  frame  breaks  down  into  10-ft  or  5-ft 
sections.  The  sections  which  support  the  small  test  specimens  will  have  a  substructure  to  provide  stability 
and  specimen  mounting  features.  Figure  B-9  shows  the  substructure  or  skeleton  of  a  typical  1 0-ft  section. 
A  5-ft  section  would  be  similar.  It  will  be  fabricated  from  square  aluminum  tubing  assembled  to  mimic  an 
aircraft  structure  of  stringers  and  ribs.  The  frame  sections  will  be  rolled  to  the  75  inch  radius  found  on 
Boeing  aircraft.  The  entire  substructure  will  be  assembled  into  a  single  unit  by  either  welding  the  structural 
members  together  or  by  using  a  structural  adhesive.  Once  assembled,  each  5-ft  section  will  weigh 
approximately  50  lbs. 


Figure  B-9.  Substructure  of  support  frame  for  small  test  specimens. 
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Fuselage  Skin.  The  test  frame  will  be  entirely  covered  with  a  thin  aluminum  skin.  Figure  B-10  shows  the 
resulting  assembly.  The  upper  and  lower  portions  of  the  skin  are  formed  by  the  test  specimens  that  are 
mounted  into  two  separate  lap  joint  inspection  row’i.  The  rows  will  be  aligned  to  match  the  lap  joint 
inspections  at  stringers  14  and  19  on  the  Boeing  737.  An  aluminum  skin  of  comparable  thickness  will  be 
used  to  fill  the  gap  between  the  two  lap  splice  inspection  rows.  All  three  substructures  used  to  mount  the 
20-in.  test  specimens  will  be  packed  in  a  single  shipping  crate. 


Figure  B-10.  Assembly  of  five-foot  test  section  with  test  specimen  and  end  supports. 

End  Supports.  Triangular  supports  will  be  placed  at  the  ends  of  each  section.  A  single  triangular  end 
support  is  shown  in  Figure  B-10.  The  three  structural  beams  (sides  of  the  triangle)  w  ill  be  hinged  at  the 
apices.  The  hinges  will  feature  a  pin  arrangement  that  allows  the  length  of  each  side  to  be  individually 
adjusted.  This  way,  the  overall  height  of  the  frame  and/or  the  angle  of  the  fuselage  section  can  be  changed; 
thus,  the  frame  can  be  reused  for  different  experiments.  The  triangular  supports  will  be  sized  to  allow  the 
inspector  to  walk  through  them.  Thus,  he/she  can  travel  the  entire  length  of  the  longitudinal  lap  splice  joint 
without  leaving  the  inspection  surface.  To  provide  sufficient  clearance  for  the  inspector  to  walk  the  length 
of  the  frame,  it  may  be  necessary  to  include  an  additional  short  leg  at  the  apex  of  the  triangle.  Figure  B-l  1 
shows  the  resulting  "dog-leg"  assembly  on  the  end  support,  as  well  as  how  the  angle  and  height  of  the 
inspection  area  can  be  adjusted  using  this  frame  design.  One  final  feature  of  note  is  the  use  of  leveling  feet 
to  allow  for  frame  assembly  on  uneven  floors.  All  of  the  triangular  supports  (six  or  seven  required  for  this 
experiment)  can  be  dismantled  and  placed  in  one  shipping  crate. 

Mounting  the  Substructures  and  Linking  them  Together.  Individual  sections  will  be  connected  to  the 
triangular  end  supports  by  using  aluminum  round  stock  as  metal  dowels  (sec  Figure  B-10).  They  will  slide 
into  the  square  aluminum  tubing  and  will  be  pinned  into  place.  These  round  bars  also  serve  the  dual 
purpose  of  connecting  the  fuselage  sections  into  a  single  fuselage  segment.  Figure  B-l  2  shows  the 
complete  assembly  with  the  test  specimens  in  one  frame  and  the  Foster-Miller  panels  in  another.  The 
handling  fixtures  that  are  currently  used  on  the  full-size,  Foster-Miller  panels  will  be  modified  as  necessary 
to  conform  to  this  set-up. 


Figure  B-l  1.  Variation  in  end  support  assembly  to  incrementally  adjust  height  and  angle  of  inspection 
surface 


20  IN.  WIDE  TEST 
SPECIMENS 
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Test  Specimen  Mounting.  The  method  used  to  mount  the  20-in.  wide  test  panels  to  the  substructure 
sections  allows  for  quick  and  easy  placement  and  provides  the  ability  to  achieve  accurate  alignment  of  the 
lap  splice  rows.  The  frame  is  designed  such  that  the  specimens  clip  into  place.  The  clips  run  along  the 
upper  and  lower  edges  of  the  specimens.  The  resulting  specimen  attachment  is  shown  in  Figure  B-13. 
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Figure  B-13.  Test  specimen  mounting  to  mock  fuselage  frame. 
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